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SECTION  I 

INTRODUCTION  AND  SUMMARY 

In  a recent  USAF  study  of  information  processing  requirements  it  was 
shown  that  in  almost  all  applications,  computer  software  was  the  major 
source  of  difficult  problems,  a major  contributor  to  operational  per- 
formance penalties,  and  potentially  the  largest  source  of  life  cycle 
cost.  During  the  acquisition  cycle  of  a weapon  system,  the  military 
spends  half  of  the  total  system  acquisition  cost  on  software  (Reference  1). 
The  Air  Force  spent  between  one  and  one  and  one-half  billion  dollars  on 
software  alone  in  1972,  which  represented  an  expenditure  of  about  four  to 
five  percent  of  the  total  Air  Force  budget  (Reference  2).  In  comparison, 
the  Air  Force  spends  only  $300  to  $400  million  per  year  on  computer  hardware. 
There  have  been  large  expenditures  for  software  packages  in  the  recent 
past,  and  yet  it  appears  that  software  development  costs  are  continually 
rising  ($6  to  $30  per  line  of  code  and  upwards  to  $150  per  line  of  code 
for  very  complex  space  systems)  (Reference  2).  In  a July  1976  "Newsweek", 
it  was  stated  that  in  the  1950’s  the  rate  of  computer  hardware  costs  to 
software  costs  was  4 to  1 , compared  to  the  present  figure  of  1 to  4,  a 
complete  turn-around  in  a little  over  20  years  (Reference  3).  Decreasing 
hardware  production  costs  and  increasing  personnel  costs,  are  partly  the 
explanation  for  this  turn  of  events.  Figure  1 is  an  estinnated  distribution 
of  the  total  portion  of  the  USAF  budget  spent  on  software.  Figure  2 
shows  the  relationship  of  hardware  to  software  costs  projected  to  the  year 
1985  (Reference  4). 

In  addition  to  the  constantly  rising  cost  for  software  development, 
software  reliability,  unresponsiveness,  and  indirect  costs  associated 
with  slippages  in  software  developments  are  of  major  concern  to  the  USAF. 

A number  of  reports  stress  the  fact  that  in  software  products  acquired  by 
the  military,  the  quality  or  "relaibility"  of  the  software  produced  is 
generally  unacceptable  (error  rates  of  over  1 error  per  100  lines  of  code 

J 

(Reference  2).  The  CCIP-85  report (Reference  5)  states  that  military 
software  is  extremely  unreliable  and  unacceptable  at  the  present  time. 

A number  of  examples  were  given  which  indicate  that  software  errors  have 
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Figure  2.  Hardware/Software  Cost  Trends  (Reference  4) 
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caused  considerable  loss  in  terms  of  hardware  equipment.  Current  Air 
Force  software  reliability  problems  indicate  that  software  errors  could 
cauue  the  Air  Force  to  lose  critical  command  post  or  satellite  capabilities 
in  a strategic  crisis  situation. 

Also,  software  is  frequently  unresponsive:  95  percent  of  the  SAC 
Automated  Conmand  and  Control  System  (465L)  software  package  delivered  to 
SAC  had  to  be  rewritten  to  meet  SAC's  operational  needs  and  67  percent  of 
the  Seek  Data  II  software  used  during  the  Vietnam  conflict  had  to  be 
rewitten  (Reference  6).  In  addition,  it  has  been  established  that  indirect 
costs  of  software  slippages  generally  far  exceed  the  direct  costs  (Reference 
7). 


The  above  examples  emphasize  the  general  criticality  of  military 
software,  where  many  operations  have  to  be  performed  in  a few  seconds  or 
less.  As  a result,  the  military  is  taking  a closer  look  at  software 
development  procedures  and  a greater  portio'’  of  its  R&D  resources  are 
being  allocated  to  the  software  verification  and  validation  (V&V)  and 
development  tools.  However,  in  order  to  reduce  the  costs  of  software, 
the  Air  Force  should  have  a defined  method  which  will  allow  the  analyst, 
software  engineer,  and/or  project  manager  to  estimate  software  development 
and- support  costs,  given  the  basic  requirements  of  the  new  system  being 
developed.  There  exists  today  computerized  models  to  predict  the  cost  of 
hardware  in  terms  of  resea»‘ch  and  development,  acquisition,  and  operation/ 
support  costs.  These  models,  some  of  which  have  been  validated,  are 
widely  used.  (NOTE:  There  are  those  who  will  disagree  that  the  military 
possesses  the  models  to  accurately  predict  hardware  costs,  but  "life  cycle 
cost"  models  for  hardware  do  exist).  Hardware  has  been  in  existence  longer 
than  software  yet  there  is  still  no  widely  agreed-upon  hardware  life  cycle 
cost  methodology.  Hence,  the  expectation  that  a validated/proven  software 
life  cycle  cost  model  will  come  along  in  the  next  few  years  may  not  be 
realistic  but  it  is  certainly  worth  working  toward. 
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The  purpose  of  this  report  is  to  investigate  the  field  of  cost 
estimating  as  applied  to  computer  software,  primarily  for  the  purpose  of 
estimating  the  total  life  cycle  cost  of  software  for  new  avionics  equipment 
under  development.  In  this  report  past,  present,  and  future  efforts  to 
derive  a valid  methodology  to  predict  software  life  cycle  costs  will  be 
discussed.  Several  methods  or  "models"  which  are  usable  today  will  be 
presented  or  referenced.  The  statistical  confidence  with  which  one  may 
use  the  methods,  however,  is  quite  low.  Therefore,  they  are  presented 
not  as  tested,  well-proven  tools,  but  as  guidelines  to  be  used  in  conjunction 
with  additional  techniques,  experience,  and  judgement.  It  is  hoped  that 
the  reader  will  acquire  some  knowledge  as  to  what  is  being  done  to  predict 
software  costs.  The  acquisition  of  software,  the  support  of  software, 
and  the  reliability  of  software  are  topics  permitting  unlimited  discussions. 
This  report  will  deal  with  each  area  in  as  much  detail  as  time  permits, 
primarily  focusing  on  the  acquisition  costs  of  software.  Due  to  limited 
time  and  space,  the  models  are  described  in  very  concise  terms  in  this 
report.  Before  actually  applying  a model,  the  reader  is  referred  to  the 
source  document  containing  a more  detailed  discussion  of  a particular 
equation/ model . 

The  following  types  of  computer  software  programs  were  analyzed  by 
various  organizations  in  an  attempt  to  derive  a software  cost  estimating 
procedure:  Management  information  systems  (MIS);  avionics;  scientific 
and  engineering;  logistic  and  maintenance;  command,  control  and  com- 
munication, and  intelligence.  The  question  of  whether  avionics  software 
is  more  costly  than  MIS  software,  etc.,  does  not  seem  to  be  addressed  to 
the  point  of  defining  a definite  relationship,  although  it  is  thought 
that  space  and  avionics  software  are  more  costly  due  to  required  testing 
and  more  detailed  design  effort. 
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SECTION  II 
DEFINITION  OF  TERMS 

Most  persons  that  will  read  this  report  are  already  familiar  with 
the  terms  that  have  and  will  be  used.  Since,  however,  most  readers  have 
their  own  definition  of  terms  such  as  software,  reliability,  operation 
and  support,  etc.,  these  terms  will  be  defined  to  insure  a common 
foundation  for  the  discussion  to  follow. 

1 . SOFTWARE 

AFAL-TR-73-341  defines  software  as  "the  programs  and  routines  used 
to  extend  the  capability  of  automatic  data  processing  equipment."  To 
expand  the  definition  of  software  as  defined  in  AFAL-TR-73-341,  this 
report  also  considers  software  as  the  programs  and  routines  used  to  extend 
the  capability  of  computers  which  are  imbedded  within  weapon  systems  (not 
merely  automatic  data  processing  equipment).  Software  is  further  broken 
down  into  two  types,  basic  software  and  application  software.  For  purposes 
of  this  report,  the  term  software  will  include  all  necessary  documentation 
from  functional  specifications  to  flowcharts  and  users'  manuals  as  well  as 
the  actual  computer  code. 

a.  Basic  software  comprises  those  routines  and  programs 
designed  to  extend  or  facilitate  the  use  of  particular  automatic  data 
processing  equipment,  the  requirements  for  which  take  into  account  the 
design  characteristics  of  such  equipment.  This  software  is  usually 
provided  by  the  original  equipment  manufacturers  and  is  normally  essential 
' to  and  a part  of  the  system  configuration  furnished  by  him.  Examples  of 
basic  software  are  executive  and  operating  programs,  diagnostic  programs, 
compilers,  assemblers,  utility  routines,  file  management  programs,  and 
data  management  programs. 
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b-  Application  software  consists  of  those  routines  and  programs 
designed  by  or  for  automatic  data  processing  equipment  users,  to  accomplish 
specific  mission-oriented  tasks,  jobs,  or  functions  using  the  automatic 
data  processing  equipment  and  basic  software  available.  Except  for  general 
purpose  packages  which  are  acquired  directly  from  software  vendors  or 
from  the  original  equipment  manufacturers,  this  type  of  software  is  normally 
developed  by  the  user  in-house  or  through  contract  services. 

2.  REAL  TIME  SOFTWARE 

A real  time  computer  software  system  is  defined  as  one  which  controls 
an  environment  by  receiving  data,  processing  them,  and  taking  action  or 
returning  results  sufficiently  quickly,  to  effect  the  functioning  of  the 
environment  at  that  time.  "Sufficiently  quickly"  refers  to  the  time 
which  "allows  users  to  interact  with  the  computer  on  a time  scale 
appropriate  for  human  beings  --  on  the  order  of  a few  seconds  between 
responses." 

3.  SOFTWARE  ACQUISITION  COST 

The  term  acquisition  cost  of  software  is  used  in  the  same  sense  as 
acquisition  cost  of  hardware.  It  includes  the  cost  of  analysis,  design, 
progrartmi ng , checkout,  test,  and  documentation. 

4.  SOFTWARE  RELIABILITY 

Software  reliability  is  the  rate  at  which  errors  are  detected  in  a 
program,  i.e.,  number  of  errors  per  unit  time  of  operations.  A more 
formal  definition  of  reliability  states  "Reliability  - the  characteristic 
of  an  item  expressed  by  the  probability  that  It  will  perform  a required 
function  under  stated  conditions  for  a stated  period  of  time."  Reference 
3).  "Reliability  is  the  measure  of  the  frequency  of  failure  of  the 
computer  software."  (Reference  5), 
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5.  MAINTAINABILITY 

Maintainability  is  defined  as  a measure  of  the  ease  with  which  errors 
in  a computer  program  can  be  corrected  and  system  function  and  capability 
can  be  expanded  or  added.  Unlike  hardware,  software  maintainability 
entails  some  "redesign." 

6.  OPERATION  AND  SUPPORT 

The  operation/ support  (OiS)  or  maintenance  costs  of  software  include 
the  costs  associated  with  using  or  "running"  the  computer  programs, 
modification  or  adaptation  of  an  existing  program  to  a computer  system 
or  to  acconmodate  changes  in  system  software,  and  the  general  day-to-day 
reprogramming  that  must  be  accomplished  to  keep  the  program  operational. 
The  operation/support  costs  are  directly  related  to  the  reliability 
(number  of  errors)  and  the  maintainability  (cost  to  fix  the  errors). 

7.  LIFE  CYCLE  COST 

The  life  cycle  cost  (LCC)  of  software  is  the  total  of  the  research 
and  development,  acquisition,  and  operation  and  maintenance  costs. 
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SECTION  III 

REQUIREMENTS  TO  FORECAST  SOFTWARE  COSTS 

The  requirements  to  forecast  software  costs  can  be  broken  down  into 
three  areas  of  concern.  The  first  and  primary  area  is,  what  are  the 
different  methods  available  today  for  deriving  a software  cost  estimate. 

Once  the  method  has  been  determined,  how  are  the  historical  data  items 
collected  to  utilize  the  method.  The  method  of  deriving  a software  cost 
estimate  plus  the  management  responsibilities  and  proposed  work  breakdown 
structure  for  the  collection  of  software  data  will  be  discussed  in  the 
following  paragraphs. 

1.  METHODS  FOR  DERIVING  SOFTWARE  COSTS 

In  October  1974,  a government/ industry  software  workshop  was  held 
at  Hanscom  AFB,  Massachusetts,  sponsored  by  Electronic  Systems  Division 
(ESD)  (AFSC).  The  purpose  of  the  workshop  was  to  "improve  coirmuni cations 
between  industry  and  government  in  the  problems  of  forecasting  software 
development  costs"  (Reference  8).  The  objective  of  agencies  dealing  in 
software  is  to  improve  the  accuracy/credibility  of  future  software  cost 
estimates  for  electronic  defense  systems.  The  workshop  listed  several 
methods  for  deriving  software  cost,  the  principal  ones  being  factors, 
experts,  ratio  to  previous  experience,  ratio  to  total  system  dollars,  and 
probabilities. 

The  factors  method  involves  identification  of  cost  drivers  and  the 
formulation  of  an  equation/series  of  equations  relating  these  drivers  to 
cost.  These  equations,  cost  estimating  relationships,  (CER's)  are 
derived  through  the  application  of  statistical  methods  to  appropriate 
historical  data.  During  the  ESD  Workshop  the  following  dominant  cost 
drivers  were  identified:  (1)  number  of  instructions  in  the  program, 

(2)  type  of  progranming  language  (Higher  Ordered  Language  (HOL),  Machine 
Ordered  Language  (MOL)),  (3)  real  time  application,  (4)  type  of  program, 

(5)  desired  quality,  (6)  amount  of  documentation,  (7)  hardware  constraints, 
(8)  schedules,  (9)  size  of  data  bases,  (10)  complexity,  and  (11)  personnel 
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and  management  functions.  Table  4 lists  the  driving  factors  the  government 
and  industry  agreed  upon  and  best  guesses  as  to  the  effects  these  drivers 
have  on  software  costs.  NOTE:  Items  1,2,  and  10  are  common  factors  seen 
in  the  majority  of  CER's  developed  thus  far.  The  factor  method  is  good 
from  the  standpoint  that  it  provides  a quantitative  relationship  which  is 
easy  to  apply.  A major  drawback  of  the  factors  method  is  that  some  inputs 
are  subjective  in  nature,  such  as  the  complexity  of  the  program,  the  skill 
level  of  progranmer,  etc.  Development  of  reliable  software  cost  estimating 
relationships  (SCER's)  using  the  factors  method  is  currently  limited  by  the 
quality/quantity  of  historical  software  cost  data  available.  The  data 
problem  and  what  is  being  done  to  overcome  it,  will  be  discussed  later. 

The  method  of  "experts"  (delphi  technique)  has  been  used  much  in 
the  past.  This  method,  as  the  name  implies,  is  dependent  upon  subjective 
opinions  of  a group  of  experts  in  the  software  field.  Results  are 
obviously  only  as  good  as  the  participants  of  the  group.  The  method  of 
experts,  very  similar  to  the  corporate  approach,  is  directed  more  along 
the  lines  of  engineering  estimates  than  statistically  derived  CER's. 

After  estimating  program  size  and  complexity  (usually  by  comparison  with 
similar  previous  programs),  historical  corporate  productivity  data  is 
applied  to  estimate  direct  labor  hours  (and  thus  direct  costs)  for  coding 
and  debugging.  Costs  for  all  other  phases  and  factors  in  the  development 
process  are  then  estimated  as  (historical  corporate)  percentages  of  this 
direct  labor  cost. 

The  two  methods  of  ratio  to  previous  experience  and  ratio  to  total 
system  dollars  both  have  the  same  characteristics.  Their  good  feature  is 
that  the  ratio  is  developed  on  real  experience  and  the  drawback  is  the 
degree  with  which  the  results  represent  the  actual  cost  incurred.  In  both 
ratio  methods  the  data  problem  appears  again;  that  is,  there  is  the  need 
for  better  data  collection  and  analysis. 

The  difficulty  of  understanding  the  procedure  used  detracts  from  the 
probabilistic  method.  This  method  reflects  reality  by  using  past  programs 
but  it  is  harder  to  sell  its  use  to  someone  'with  no  understanding  of 
probability  theory.  Again,  to  develop  "good"  best  fit  probability  models, 
the  need  for  data  is  ever  present. 
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2.  MANAGEMENT  VISIBILITY 

Management  control,  visibility  of  the  software  structure,  and  a 
standardized  framework  for  collecting  historical  costs,  sizing  and 
requirement  data,  is  greatly  needed  by  the  Air  Force  today.  A point 
stressed  during  the  ESD  Workshop  (Reference  8)  is  the  need  for  good 
specifications  early  in  the  program.  "A  good  specification  is  fundamental 
to  building  a realistic  software  cost  estimate."  Costing  in  the  Air  Force 
Avionics  Laboratory  and  throughout  th**  R&D  coimunity  is  done  at  various 
stages  in  the  system  life  cycle,  but  the  original  most  crucial  costing  is 
done  in  the  conceptual  phase.  During  this  timeframe  high  level  management 
often  wants  to  be  able  to  make  a "go  or  no  go"  decision  as  to  system 
development,  usually  based  on  a cost-benefit  analysis  (costs  to  develop, 
deploy,  and  maintain  the  system  versus  the  benefits  to  be  gained  by 
acquisition  of  the  system).  In  these  early  stages  the  costing  study  is 
extremely  difficult  because  system  documentations  are  usually  incomplete 
or  lacking  in  detail  and  may  be  inconsistent  or  ambiguous.  The  appropriate 
time  at  which  an  initial  software  cost  estimate  should  be  attempted  is  at 
the  earliest  possible  point  after  functional  design  specifications  are 
completed  (Part  I Specifications).  Once  the  functions  and/or  modes  the 
software  program  will  deal  with  are  known,  the  "size  of  your  program" 
and  a "rough"  estimate  of  the  software  cost  can  be  developed  from  analysis 
of  the  functions  of  the  program.  The  cost  of  deriving  a good  software 
cost  estimate  is  high  since  much  preliminary  design  work  is  required. 
However,  the  software  workshop  agreed  that  in  order  to  accurately  predict 
software  costs,  a considerable  amount  of  design  work  and  project  planning 
must  have  been  accomplished. 

According  to  industry  comments,  the  majority  of  software  cost  estimates 
are  obtained  from  elementary  sizing  parameters  of  the  "estiinated  number  of 
instructions"  usually  derived  from  historical  experience  and/or  engineering 
judgement.  Once  the  size  of  the  program  is  determined,  then  manpower 
requirements  are  estimated,  leading  to  the  cost  of  the  software  package 
development.  Both  rule  of  the  thumb  and  mathematical  methods  do  exist 
but  none  are  very  reliable  or  validated. 
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3.  WORK  BREAKDOWN  STRUCTURE 

A major  problem  in  the  process  of  comparing  or  tracking  actual  software 
development  cost  is  that  there  is  a lack  of  a coimon  language,  methodology, 
and  work  breakdown  structure  (WBS)  which  would  provide  a basis  for  developing 
and  comparing  cost  estimates.  In  a report  prepared  for  NAVSEA,  "Interim 
Guidance  for  Preparation  of  Cost  Estimates  for  Tactical  Software  Programs," 
Oct  74,  an  interim  work  breakdown  structure  was  provided  as  the  frame-of- 
reference  for  all  NAVSEA  tactical  software  program  cost  estimation 
’><•  (Reference  9).  The  report  provided  a brief  description  of  a typical 

software  development  process  and  how  the  various  activities  relate  to  the 
work  breakdown  structure.  Worksheets  or  summary  report  formats  were 
presented  to  cover  each  of  the  following  topics: 

a.  Cost  Estimate  for  XXX  Tactical  Software. 

b.  Cost  Incurred  Schedule. 

c.  Tactical  Software  Program  Sumnary 

d.  Milestone/Resource  Allocation 

The  problem  and  the  inability  to  apply  WBS  methods  to  software 
development  can  best  be  sunned  up  in  a quote  from  the  NAVSEA  Report: 

"In  the  past,  industry  and  NAVSEA  project  managers/engineers  have  not 
been  able  to  describe  or  define  the  software  programs  for  a tactical 
system  at  levels  parallel  to  that  which  have  been  developed  for  technical 
management  of  hardware.  This  inability  to  develop  realistic  work  packages 
and  milestones  for  management  of  software  programs  has  resulted  in 
ineffective  monitoring  and  cost  forecating.  In  addition,  the  lack  of  a 
suitable  connon  structure  of  WBS  language  has  limited  the  development  of 
norms  and  valid  data  banks  for  transferring  experience  and  interfacing 
with  industry."  (Reference  9) 

It  is  quite  possible  that  the  approach  of  the  NAVSEA  report  has  been 
adopted  in  MIL-STD-881,  but  it  appears  that  the  WBS  has  not  yet  been  fully 
utilized  by  managers  of  software  packages,  which  leaves  the  cost  estimators 
in  a state  of  flux. 
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3.  DISTRIBUTION  OF  SOFTWARE  DEVELOPMENT  EFFORT 

A majority  of  the  models  presented  in  Section  V of  this  report  arrive 
at  a total  cost  for  software  development.  A distribution  of  software 
development  effort  or  allocation  of  effort  during  this  phase  is  sometimes 
desired.  The  development  process  can  be  broken  down  into  three  major 
phases;  Analysis  and  Design,  Coding  and  Debugging,  and  Integration  and 
Test.  Documentation  costs  will  be  included  in  Integration  and  Test. 

Table  1 represents  some  findings  into  how  the  three  phases  are  distributed 
as  a percent  of  the  development  effort.  A general  consensus  of  a 40,  20, 
40  percent  distribution  can  be  drawn  (i.e.,  40%  for  Analysis  & Design, 

20%  for  Coding  and  Debugging,  and  40%  for  Integration  and  Test).  If  only 
airborne  and  space  programs  are  considered,  it  can  be  seen  that  more 
emphasis  is  being  given  to  Analysis  and  Design  and  a great  deal  to 
Integration  and  Test.  In  airborne  programs,  coding  and  debugging  are  a 
smaller  part  or  percent  of  the  effort. 
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TABLE  1 

DISTRIBUTION  OF  SOFTWARE  DEVELOPMENT  EFFORT 


SOURCE 

PROGRAM/COMPANY 

ANALYSIS 

AND 

DESIGN 

PERCENTAGE 

AND 

DEBUGGING 

INTEGRATION 

AND 

TEST 

Ref  2 

SAGE/NTDS 

35  % 

17  % 

48  % 

Ref  2 

TRW{ COMMAND/CONTROL) 

46 

20 

34 

Ref  2 

GEMINI/SATURN 

34 

20 

46 

Ref  2 

OS/ 360 

33 

17 

50 

Ref  2 

TRW  (SCIENTIFIC) 

44 

26 

30 

Ref  10 

RAYTHEON  (BUSINESS) 

44 

28 

28 

Ref  10 

INFORMATIES  CORP 

46 

16 

48 

Ref  11 

TITAN  III 

33 

28 

39 

Ref  11 

X-15 

36 

17 

47 

Ref  11 

APOLLO 

31 

36 

33 

Ref  11 

GEMINI 

36 

17 

47 

Ref  11 

SATURN  V 

32 

24 

44 

Ref  11 

AIRBORNE  DAIS  (EST.) 

38 

15 

47 

Ref  11 

GRC  EXPERIENCE 

30 

20 

50 

Ref  11 

SKYLAB 

38 

17 

45 

Ref  11 

TRW 

40 

20 

40 

Ref  11 

SETS/BL 

42 

18 

40 

Ref  12 

IBM 

30 

40 

30 

Ref  12 

AEGIS 

38 

26 

36 

Ref  12 

AN/BQQ-B 

31 

43 

26 

Ref  13 

COST-BY-FUNCTION  MODEL 

34.5 

18.0 

47.5 

AFAL  (AAA-3) 

38.7 

21.7 

39.6 

AVERAGE 

36.8% 

22.9% 

40.7% 
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SECTION  IV 

PAST/PRESENT  AND  FUTURE  STUDIES  (STATE  OF  THE  ART) 

The  alarming  increase  in  software  development  costs  and  the 
decreasing  dollar  resources  available  for  software  development  have 
forced  the  government  and  especially  the  military  to  find  new  ways  of 
producing  quality  software  within  very  limiting  constraints.  These  costs, 
coupled  with  the  fact  that  the  quality  of  software  produced  is  generally 
unacceptable  (error  rates  of  over  1 error/100  lines  of  code),  has  spurred 
several  Air  Force  organizations  into  conducting  various  studies  on  software 
cost  estimating  procedures  and  development  methods. 

1.  ROME  AIR  DEVELOPMENT  CENTER  (RADC)  EFFORTS 

The  Rome  Air  Development  Center  has  begun  an  extensive  study  of  the 
software  development  field.  The  goal  of  the  RADC  program  is  to  achieve 
higher  quality,  lower  priced  software  through  the  reduction  of  intrinsic 
error  rates;  the  improvement  of  programmer  productivity  (by  using  better 
prograitming  languages,  better  design,  coding  and  testing  techniques,  and 
better  management  control)  and  improvement  of  the  readability,  portability, 
documentation,  and  maintainability  of  software  code. 

To  achieve  the  Air  Force  objectives  in  the  software  development  area, 
the  RADC  approach  is  to  develop  a system  of  software  facilities,  all  linked 
to  a centralized  software  data  base.  Three  facility  development  efforts 
are  currently  under  negotiation  at  RADC  for  facilitating  development  of 
the  automated  analysis  system.  The  first  is  a multiregression  facility 
for  statistically  correlating  various  data  with  respect  to  reliability, 
cost,  and  productivity.  The  second  is  a facility  to  use  RADC's  On-line 
Pattern  Analysis  and  Recognition  System  (OLPARS)  to  use  the  pattern 
recognition  for  determining  software  reliability.  The  third  is  a language 
control  facility  for  collecting  various  information  on  language  usage  and 
relating  errors  to  specific  language  constraints. 
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Principal  functions  of  the  centralized  software  data  base  will  be 
to:  (1)  collect  software  production  data,  (2)  provide  a comouterized 
data  base  of  raw  data  for  reliability,  cost  and  productivity  analysis, 

(3)  provide  analytical  tools  for  data  analysis  and  modeling,  and 

(4)  generate  standard  reports  on  information  contained  in  the  repository. 

The  necessity  for  collecting  software  data  is  based  on  the  fact  that  no 
clear  conclusions  or  predictions  can  be  made  about  quantities  of  interest, 
such  as  the  reliability  of  software  or  an  accurate  estimation  of  software 
production  costs,  without  the  historical  data  base.  The  repository  data 
related  to  the  design,  coding,  testing,  and  maintenance  of  software  will 
include  software  error  data,  software  cost,  complexity  and  productivity 
data,  and  computer  language.  Once  centralized  software  data  bases  have 
been  established,  a number  of  efforts  can  proceed.  For  example,  information 
on  the  cost  of  software  developments  as  related  to  the  techniques  used  to 
develop  the  software  and  on  error  rates  as  a function  of  a particular 
language  or  a language  feature  are  important  areas  of  investigation  that 
will  be  aided  significantly  by  the  repository. 

A major  requirement  for  effective  use  of  the  software  data  base  is 
a software  modeling  facility.  This  type  of  facility  would  enable 
researchers  to  model  various  aspects  of  the  software  development  process. 

For  example,  cost  estimating  models  based  on  factors  like  functional 
requirements,  problem  complexity,  and  programming  experience,  and  sizing 
models  for  determining  necessary  computer  resources,  are  critical  for 
accurate  software  cost  estimates.  Reliability  models  for  predicting  the 
occurrence  or  number  of  software  errors  in  operational  software  are 
important  in  answering  questions  related  to  the  release  of  software  for 
operational  use  and  the  progress  of  software  testing.  These  models, 
plus  software  complexity  and  production  models,  can  lead  to  a true 
appreciation  of  the  significant  factors  underlying  software  development, 
thus  leading  to  increased  control  over  the  software  process.  Contractors 
involved  in  RADC  software  programs  include  System  Development  Corporation 
(SDC),  Illinois  Institute  of  Technology  Research  Institute  (IITRI), 
Polytechnic  Institute  of  New  York  (PINY),  and  the  MITRE  Corporation. 
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The  System  Development  Corporation  (SDC)  is  conducting  a data 
collection  study  for  RADC.  SDC  is  studying  the  general  area  of  data 
collection,  storage,  and  retrieval.  Major  emphasis  has  been  placed  on 
the  problems  related  to  the  collection  of  software  production  data.  They 
also  analyzed  previous  efforts  by  IBM,  TRW,  and  MITRE  (Reference  1)  in 
the  software  data  collection  area. 

The  Illinois  Institute  of  Technology  Research  Institute  (IITRI)  is 
under  contract  with  RADC  to  investigate,  among  other  things,  the  specifi- 
cations for  a pilot  repository  facility  at  RADC.  Thus,  SDC  and  IITRI  are 
working  closely  to  develop  specifications  for  a Software  Data  Collection 
& Repository  System  at  RADC. 

The  MITRE  Corporation  has  developed  models  for  measuring  structural 
complexity  of  software,  and  is  currently  developing  a Software  Implemen- 
tation Monitor  (SIMON)  for  RADC  to  automatically  gather  and  analyze  data 
during  software  development. 

Polytechnic  Institute  of  New  York  (PINY)  is  currently  investigating 
modeling  techniques,  similar  to  those  used  for  hardware  reliability,  for 
use  in  the  software  area.  For  example,  PINY  has  developed  a model  for 
predicting  the  reliability  of  software  based  on  the  use  of  Markovian 
processes  to  determine  the  probability  that  a given  software  system  is 
in  either  an  "up"  state  (no  errors  present  in  the  system)  or  a "down" 
state  (an  error  has  occurred  and  is  being  corrected).  Other  areas  which 
PINY  is  investigating  at  this  time  include: 

a.  Models  to  measure  software  complexity  and  develop  relationships 
among  error  content,  debugging  effort,  program  size  and  run  time. 

b.  A study  of  the  effects  of  modular  and  structured  programming  on 
program  errors, 

c.  Models  to  test  effectiveness  in  removing  software  errors. 
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d.  Development  of  models  for  comparison  of  different  programming 
languages  with  respect  to  such  features  as  core  size,  run  time,  development 
test,  and  debugging  costs. 

With  respect  to  software  reliability/maintainability,  RADC  is  currently 
planning  an  effort  to  develop  software  reliability  models  based  on  the 
Bayesian  statistical  theory.  Along  with  this,  methods  will  be  developed 
for  making  acceptance  or  rejection  decisions  about  a software  package 
during  software  testing  using  these  Bayesian  models  and  Bayesian  techniques. 
Also  planned  by  RADC  is  the  evaluation  of  computer  programs  and  designs 
in  terms  of  a quantitative  measure  of  maintainability,  and  the  restructuring 
of  computer  programs  for  reliability  and  maintainability  improvement.  As 
part  of  this  effort,  a maintainability  model  will  be  developed  which  tracks 
and  measures  the  propagation  of  modifications  and/or  errors  through  a 
system  of  software  modules,  thus  leading  to  a measurement  of  maintainability. 

2.  ELECTRONIC  SYSTEMS  DIVISION  (ESD)  EFFORTS 

Two  Electronic  System  Division  (ESD)  organizations,  ESD/MCIO  and 
ESD/ACCI,  are  conducting  studies  analyzing  the  software  development  cycle 
and  predicting  software  costs. 

On  1-2  October  1974  an  ESD  workshop  entitled.  Government/ Industry 
Software  Seizing  and  Costing  was  held  at  Hanscom  Air  Force  Base,  Bedford, 
Massachusetts.  The  primary  output  of  the  workshop  was  a list  of  factors 
and  details  on  how  these  factors  affect  the  cost  of  software  development. 

This  list  is  what  is  referred  to  in  this  report  as  the  "ESD  Model",  and 
is  described  in  detail  in  Section  V-3.  The  dominant  factors  affecting 
software  costs  were  identified  as  the  number  of  instruction,  programming 
language,  real  time  application,  type  of  program,  desired  quality,  amount 
of  documentation,  hardware  constraints,  schedules,  size  of  data  base, 
complexity,  stability  of  requirements,  and  personnel  and  management 
required.  Certain  procurement  procedures  such  as  subordination  of  software 
design  goals  to  hardware  design  goals  also  were  identified  as  having  a 
decided  effect  on  software  costs.  The  workshop  agreed  upon,  one  general 
point,  i.e.,  t.hat  deriving  a good  software  cost  estimate  is  very  expensive. 
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Another  point  of  interest  was  the  need  for  an  improved  work  breakdown 
structure  (WBS)  and  the  fact  that  MIl-STD-881  has  not  been  fully  utilized 
to  decompose  complex  software  projects  into  manageable  work  packages. 

To  further  the  software  cost  estimating  techniques  that  exist  today, 
ESD/MCI  has  two  major  efforts  at  the  present  time  (Reference  6).  The  first 
is  a study  entitled  "Life  Cycle  Costing  of  System  Software/Computer 
Resources,"  being  conducted  by  General  Research  Corporation  (GRC).  The 
objective  of  the  study  is  to  develop  WBS  down  to  a level  sufficient  to 
identify  software  cost  elements  and  functional  requirements.  The  second 
step  of  this  effort  will  be  to  collect  data  against  this  WBS.  After  these 
two  steps  have  been  taken,  GRC  will  employ  statistical  methods  to  develop 
CIRs  relating  previously  identified  cost  elements  to  resource  expenditure 
for  each  phase  of  the  software  life  cycle.  A follow  on  effort  entitled 
"Software  Cost  Prediction  Aids"  will  render  these  CERs  compatible  with  a 
generalized  life  cycle  cost  model  like  the  MITRE  Electronic  Systems  Cost 
Model,  or  a functional  description  tool  like  the  Computer  Aided  Require- 
ments Analyses  (Reference  14).  Development  of  a total  life  cycle  cost 
model  for  software  is  expected  to  be  completed  in  October  1976.  The 
results  of  the  GRC  effort  under  contract  FI 9628-76-C-0180  are  documented 
in  a preliminary  draft  report  entitled  "Cost  Reporting  Elements  and 
Activity  Cost  Tradeoffs  for  Defense  System  Software."  The  six  month 
study  investigated  the  problems  of  software  cost  estimation,  hypothesizing 
relationships,  gathering  and  analyzing  data,  and  examining  reporting 
systems.  There  exists  equations  relating  cost  (in  terms  of  estimating 
the  man-months)  to  the  different  phases  of  the  software  life  cycle.  The 
equations  are  not  presented  here  since  it  would  be  worthwhile  to  wait  for 
the  final  report  to  be  released.  The  following  quote  exhibits  the 
confidence  of  the  derived  relationships:  "Our  second  major  objective 
was  to  develop  improved  software  cost  estimating  relationships.  A 
significant  amount  of  work  had  been  previously  devoted  to  this  task.  The 
work  was  performed  by  competent  groups  and  focused  on  estimating  total 
man-hours  or  costs.  Results  have  been  disappointing,  with  derived  re- 
lationships exhibiting  large  variance."  Some  of  the  major  findings  by 
GRC  Include  the  following. 

a.  Accurate  estimating  relationships  for  each  life  cycle  phase 
cannot  be  developed  independent  of  the  other  phases 
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b.  Estimating  the  tradeoffs  between  the  life  cycle  phases  is  of 
prime  importance. 

c.  Estimating  the  tradeoffs  can  also  lead  to  the  development  of 
rules  for  optimal  allocation  among  life  cycle  phases. 

The  6RC  study  to  date  is  the  most  current  and  complete  effort  accomplished 
to  determine  operation  and  support  cost  of  software. 

The  second  major  effort  is  the  Air  Force  Software  Library.  ESD/MCI 
has  developed  a software  library  which  is  presently  on-line  in  prototype 
form  at  ASD?  The  library  is  designed  to  collect  technical  data  on  existing 
software  packages.  The  library  at  the  present  time  contains  a description 
of  the  software  program  and  the  person  or  organization  to  contact  for 
additional  information.  An  effort  is  now  underway  to  collect  and  store 
data,  where  available,  on  resources  expended  in  developing  and  maintaining 
these  programs. 

In  a draft  of  a proposed  in-house  research  program  for  improving 
software  cost  estimating,  ESD/ACCI  proposes  to  develop  two  models,  one 
a "robust  regression  model",  the  second  a "software  development  simulator" 
(Reference  14). 

The  robust  regression  model  will  be  developed  to  handle  small  sample 
sizes.  AFSCM  173-1  describes  the  ground  rules  and  methods  of  utilizing 
the  linear  statistical  model  as  a standard  technique  for  developing 
estimating  relationships.  AFSCN  173-1  adopts  the  principle  of  least 
squares  which  is  based  on  possessing  a normal  or  "bell-shaped"  statistical 
distribution.  When  the  variations  are  not  distributed  normally,  these 
properties  cannot  be  proven  to  be  true.  Furthermore,  AFSCM  173-1  states 
that  "when  sufficient  data  points  are  available,  the  distribution  of 
sample  means  will  remain  normal  to  a satisfactory  degree  of  approximation." 
According  to  the  ES0,''ACCI  draft  report,  30  data  points  are  generally  a 
large  enough  sample  for  the  normality  assumption  to  be  sufficiently 
approximated.  In  the  area  of  software  cost  estimating,  data  definition 
and  collection  problems  do  not  generally  give  us  30  data  points,  (homogenous 
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sample  sizes).  "Thus,  there  is  no  reason  to  believe,  a priori,  that 
standard  statistical  techniques  will  produce  accurate  CER’s,  and  the 
failures  of  SDC,  GRC,  and  Tecolati  are  not  all  that  surprising." 

(Reference  14)  The  "robust  regression  technique"  suggested  by  Capt  Bourdon 
(Reference  14)  to  handle  this  data  problem  is  to  employ  the  Kurtosis 
(fourth  standardized  moment)  of  the  least  squares  residuals.  This  method 
has  been  demonstrated  for  the  simple,  two-variable  model  and  ouite  possibly 
can  be  applied  to  a multivariate  model.  The  Kurtosis  then  can  be  used  when 
random  variables  are  not  known  to  be  distributed  normally.  According  to 
Capt.  Bourdon,  this  method  is  as  good  or  better  than  least  squares 
regression  down  to  sample  size  of  four. 

The  second  model  proposed  is  the  software  development  simulator  which 
would  employ  a Monte  Carlo  sampling  technique.  The  simulator  envisioned 
is  predicated  on  the  hypothesis  that  cost  is  an  explicit  function  of  the 
time  usage  of  direct  labor  and  computer  hours.  An  estimation  of  the 
labor  and  computer  hours  consumed  as  a function  of  time  can  be  made  and 
in  turn  the  cost  arrived  at  by  applying  standard  factors  for  engineering 
management,  overhead,  etc.  "Thus  the  simulator  serves  to  generate  and 
tabulate  statistics  on  the  consumption  of  labor  and  computer  resources 
by  simulating  the  software  development  process."  (Reference  14) 

ESD/ACC  pointed  out  in  Reference  14  that  there  is  a definite  need 
for  an  ad  hoc  planning  group  to  guide  future  research  in  the  software 
cost  estimating  area,  to  eliminate  duplication  of  research  dollars  spent 
and  time  and  effort  devoted  to  similar  tasks.  ESD/ACC  recomnends  that 
they  be  designated  the  software  cost  estimating  research  focal  point  for 
all  activities  in  the  area  conducted  by  agencies  under  the  operational 
control  of  the  ESD  Conmander.  The  proposed  ad  hoc  planning  group  would 
be  responsible  for: 

a.  Defining,  monitoring,  and  reporting  on  the  progress  of  all 
in-house  and  contracted  efforts  aimed  at  bettering  the  ability  to  predict 
the  cost  of  software. 
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b.  Coordinating  on  all  Statements  of  Work  and  serving  on  all  source 
selection  boards  for  procurements  associated  with  software  cost  estimating 
methodologies. 

c.  Organizing  and  coordinating  periodic  symposia  and  seminars  for 
the  exchange  of  data,  ideas,  and  findings  between  government,  industrial, 
and  academic  institutions  active  in  software  development  and  cost 
estimating, 

3.  TRW  EFFORTS 

TRW  is  currently  investigating  the  types  of  errors  which  are  made 
most  frequently  and  an  error  classification  scheme.  TRW  is  also  analyzing 
how  personnel,  hardware  problems,  hardware  interfaces,  operational  timing, 
and  input  requirements  contributed  to  errors  which  occurred.  TRW  has 
developed  a Mathematical  Theory  of  Software  Reliability  (MTSR)  for 
predicting  the  reliability  of  software  systems  based  on  the  complete  set 
of  possible  data  input,  values,  and  the  logical  structure  of  the  component 
modules.  They  are  currently  applying  this  theory  to  an  actual  software 
development  project  and  are  further  analyzing  the  theory  to  determine  the 
effects  of  errors  removal  on  reliability  and  the  variance  in  sampling 
techniques  for  measuring  the  reliability, 

4.  RCA  "PRICE"  SOFTWARE  ACQUISITION  MODELS 

RCA  is  currently  developing  a software  acquisition  cost  model 
analogous  to  the  proprietary  hardware  acquisition  model.  At  the  present 
time,  numerous  DoD  and  contractor  organizations  utilize  the  PRICE  model 
to  develop  cost-estimates  for  new  hardware  equipment/ systems,  in  terms  of 
development  and  acquisition  costs.  Business  Week,  7 June  1976,  "RCA's 
Uncanny  System  for  Estimating  Costs"  states  that  "Altogether,  Systems 
Conmand  will  spend  $200,000  on  PRICE  this  year.  Some  Air  Force  procurement 
agencies  even  require  that  bidders'  proposals  include  data  in  the  specific 
form  need  for  PRICE  input."  Over  the  past  two  or  three  years,  RCA  has  been 
working  to  develop  a software  cost  estimating  technique  that  is  similar  to 
the  RCA  PRICE  model.  It  would  be  unfair  to  the  reader  to  try  to  predict 
the  format,  or  the  accuracy  of  the  model  at  this  time,  but  based  on  the 
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hardware  model,  PRICE,  it  is  safe  to  state  that  this  model  looks  promising 
with  respect  to  prediction  of  software  development  costs.  The  extent  to 
which  the  new  software  cost  estimating  model  will  be  used  deoends  upon 
the  cost  charged  the  user  by  RCA  and  validation  results.  The  model  should 
appear  on  the  market  for  use  sometime  during  the  sutimer  of  1977. 
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SECTION  V 

SOME  SOFTWARE  DEVELOPMENT  COST  ESTIMATING  MODELS 

In  this  section,  several  software  cost  estimating  models  are  presented. 

It  is  generally  agreed  that  no  software  cost  estimating  relationship  (SCER) 

or  model  has  been  adequately  validated.  Hence,  the  use  of  these  models 

must  be  viewed  in  this  light.  Section  VI  includes  demonstrations  of  the 

use  of  each  of  these  models  on  actual  software  development  programs  but 

it  should  not  be  viewed  as  a validation  or  even  an  evaluation  study. 

Actual  costs  of  software  development  programs  required  for  such  an 

evaluation  were  not  available.  (Only  two  computer  programs  analyzed  have 

actual  costs  associated  with  them).  The  majority  of  models  considered 

are  based  upon  an  initial  estimate  of  the  number  of  instructions  to  be 

written  (sometimes  arrived  at  by  estimation  of  the  number  of  functions 

of  a software  program  and  general  information  as  to  number  of  instructions 

per  function).  This  implies  that  even  if  the  SCER  were  to  have  say  an 
2 

r » 0.95,  the  number  of  instructions  estimated  or  "guessed"  drives  the 
SCER  output  total  cost.  (NOTE;  r^  refers  to  the  coefficient  of 
determination  which  is  a measure  of  dispersion  showing  the  proportion  of 
total  variance  accounted  for  by  the  estimating  relationship).  Another 
general  observation  about  the  models  is  the  fact  that  most  were  developed 
on  relatively  small  data  bases  (as  small  as  two  programs  and  as  large  as 
169  programs). 

1 . THE  WOLVERTON  MODEL 

In  November  1973,  Ray  W.  Wolverton  of  TRW  Systems  Group  presented  a 
paper  entitled  "The  Cost  of  Developing  Large  Scale  Software"  (Reference 
10).  This,  most  probably,  was  not  the  first  attempt  at  a software  cost 
estimating  model,  but  has  become  the  most  widely  referenced  work  on 
software  cost  estimation.  The  Wolverton  model  Is  the  most  widely  used 
and  accepted  software  cost  estimating  technique  developed  thus  far.  This 
methodology  Is  applicable  to  large  scale  software  development  programs 
which  utilize  a "structured  progranmlng"  design  approach.  Structured 
programnlng  Implies  modular  form. 


24 


AFAL-TR-77-66 


The  basis  for  the  model  is  a TRW  proprietary  data  base  containing 
historical  information  in  the  form  of  cost  per  instruction.  Wolverton 
assumes  that  the  development  cost  varies  proportionately  with  the  number 
of  instructions.  For  each  identified  routine,  the  procedure  combines  a 
user  supplied  estimate  of  the  number  of  object  instructions,  category, 
and  relative  degree  of  difficulty  with  relationships  based  on  the  historical 
data  base  to  determine  a trial  estimate  of  the  total  software  development 
cost. 


The  first  step  in  the  procedure  is  to  estimate  the  number  of 
Instructions  in  each  category.  The  categories  which  Wolverton  defines 
are  as  follows: 

a.  (C)  Control  routine,  which  controls  execution  flow  and  is 
non-time  critical . 

b.  (I)  Input/output  routine,  which  transfers  data  into  or  out  of 
the  computer 

c.  (P)  Pre-  or  post-algorithm  processor,  which  lipulates  data 
for  subsequent  processing  or  output. 

d.  (A)  Algorithm,  which  performs  logical  or  mathematical  operations. 

e.  (D)  DaU  management  routine,  which  manages  data  transfer  within 
the  computer. 

f.  (T)  Time  critical  processor,  which  if,  a highly  optimized  machine 
dependent  code. 

To  obtain  a relative  degree  of  difficulty  there  are  basically  two 
substeps  involved.  First,  determination  of  whether  or  not  the  routine 
is  an  "old"  or  a "new"  program.  Once  that  determination  has  been  made 
(old  or  new),  then  the  program  must  be  classified  as  to  whether  it  is  an 
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easy,  medium,  or  a hard  program  to  code/design.  Therefore,  the  possible 
degrees  of  difficulty  are: 


Program 

Easy 

Medium 

Hard 

Old 

OE 

OM 

OH 

New 

NE 

NM 

NH 

where  0 = old,  N = new,  E = easy,  M = medium,  and  H = Hard.  The  results 
of  step  one  are  then  multiplied  by  the  significant  cost  per  instruction 
(CPI)  expected  for  the  type  and  difficulty  categories.  The  total  expected 
cost  of  the  program  is  the  sum  of  the  above  calculations.  Table  2 is  a 
breakdown  of  the  Category/Type  and  the  cost  per  word  expected.  As  Capt. 
Gaumer  pointed  out  in  his  thesis,  the  costs  associated  with  Wolverton's 
categories  were  extracted  from  actual  historical  costs  incurred  by  TRW, 
Inc.  based  on  1972  dollars  (Reference  15).  Using  Capt.  Gaumer 's  method 
and  the  Implicit  Price  Deflators  index  listed  in  the  "Survey  of  Current 
Business",  Wolverton/'s  1972  figures  are  multiplied  by  an  inflation  factor 
of  1.27.  Hence,  the  figures  in  Table  2 are  in  1976  dollars. 

TABLE  2 

CATEGORY/TYPE  VS  TYPE  PER  WORD 


OLD 

C 

I 

P 

A 

D 

T 

E 

$27 

$23 

$22 

$19 

$30 

$95 

M 

34 

30 

29 

25 

39 

95 

H 

38 

34 

33 

28 

44 

95 

NEW 

C 

I 

P 

A 

D 

T 

E 

$42 

$36 

$36 

$30 

$47 

$95 

M 

51 

44 

43 

38 

58 

95 

H 

62 

55 

53 

44 

71 

95 
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The  major  pitfall  with  the  Wolverton  model  lies  in  the  initial  estimation 
of  the  numbers  of  instructions  by  degree  of  difficulty  and  category.  Once 
these  estimates  are  obtained  the  model  is  easily  applied.  Results  naturally 
depend  on  the  accuracy  of  the  initial  estimates. 

2.  MODIFIED  WOLVERTON  MODEL 

The  System  Evaluation  Group,  of  the  Air  Force  Avionics  Laboratory 
developed  k computerized  version  of  the  Wolverton  Model  for  rapid  analysis 
of  software  development  costs.  As  the  title  suggests,  this  model  is  based 
on  the  TRW  work  conducted  by  Ray  Wolverton. 

The  only  required  input  to  the  computer  program  is  the  number  of 
instructions  by  type  (i.e.,  number  of  C,I,P,A,D,  and  T instructions)  as 
defined  in  Section  V-1.  The  program  utilizes  ten  equations  to  obtain  the 
cost  per  instruction  for  each  type.  These  equations  were  obtained  through 
regression  analysis  using  the  data  displayed  in  Figure  12  of  Reference  10. 
The  cost  for  time  critical  processor  type  (T)  instructions  is  assumed 
constant  as  in  the  Wolverton  Model.  Costs  associated  with  the  level  of 
effort  are  computed  as  follows:  (1)  total  cost  of  the  program  is  calculated 
from  the  number  of  instructions  and  cost  per  instruction  by  type; 

(2)  analysis  is  20  percent  of  total  cost,  design  is  18.7  percent, coding 
is  21.7  percent,  testing  is  28.3  percent  and  documentation  is  11.3  percent. 

The  Modified  Wolverton  Computer  Program  generates  program  development 
costs  for  "new"  and  "old"  code,  for  programs  ranging  in  "percent  difficulty" 
from  10-90  percent.  The  user  must,  based  on  subjective  decisions  relative 
to  these  characterizations,  select  the  appropriate  cost  figure  from  the 
spectrum  of  data  generated.  For  the  Modified  Wolverton  Model,  Wolverton’s 
categories  of  easy,  medium,  and  hard  are  redefined  as  "percent  difficulty" 
on  a scale  of  10  to  90  percent.  The  relationship  between  these  categori- 
zations is  presented  in  Table  3 below. 

TABLE  3.  PERCENT  DIFFICULTY 

Medium  Hard 

40-50-60* 


E«sy 

10-20-30* 
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The  following  characterizations  of  easy,  medium,  and  hard  programming 
tasks  developed  by  IBM  may  assist  the  user  in  assigning  "percent  difficulty" 
figures  when  utilizing  the  Wolverton  or  modified  Wolverton  models. 

a.  Easy:  Very  few  interactions  with  other  system  elements.  The 
class  includes  most  problem  programs  or  "application"  programs.  Any 
program  whose  main  function  is  to  solve  mathematical  or  logical  problems 
is  probably  in  this  class.  Easy  programs  generally  interact  only  with 
input/output  programs,  data  management  programs,  and  monitor  programs. 

b.  Medium:  Some  interactions  with  other  elements.  In  this 
category  are  most  utilities,  language  compilers,  schedules,  input/output 
packages,  and  data  management  packages.  These  programs  interact  with 
hardware  functions,  with  problem  programs,  with  monitor,  and  with  others 
in  this  class.  Complicated  by  being  generalized  enough  to  handle  multiple 
situations:  I/O  from  many  different  I/O  devices  or  management  of  class 
files  with  variable  number  of  indices. 

c.  Hard:  Many  interactions  with  other  system  elements.  All 
monitors  and  operating  systems  fall  into  this  class  because  they  interact 
with  everything.  Special  purpose  programs,  such  as  a conversational 
message  processor,  may  be  in  this  class  if  they  modify  the  master  operative 
system. 

The  Modified  Wolverton  computer  program  listing  and  sample  inputs 
and  outputs  are  contained  in  Appendix  A. 

3.  ESD  MODEL 

The  sumnary  notes  of  the  October  1974  Electronic  Systems  Division 
sponsored  software  workshop  (Reference  8)  form  the  basis  for  what  is 
referred  to  herein  as  the  "ESD  Model".  Factors  Identified  as  impacting 
software  costs  are  provided  in  Table  4. 


■A 
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TABLE  4 

FACTORS  INFLUENCING  SOFTWARE  COST 


FACTOR 

RELATION  TO  COST 

Number  of  delivered  source  instructions 

Linear,  modified  by  other  factors 

Language 

HOL:  $6-1 2/source  instruction 

MOL:  $12-24/source  instruction 

Real-time  application 

RT:  $30-60/source  instruction 

Type  (OS,  application,  utility) 

If  OS,  multiply  by  2.5 

Point  on  learning  curve 

If  unfamiliar,  multiply  by  1.5  - 2.0 

Application  area  (MIS,  avionics,...) 

Sometimes,  as  percentage  of  total  system 
cost 

"Man-rated":  test  cost  40%  of  total 

Non-"man-rated":  test  cost  'v  15%  of 
total 

Turnaround  time 

Approximately  linear  relation  to  testing 
cost 

Amount  of  documentation 

Approximately  10%  of  total;  $35  - 150/ 
non-automated  page 

Hardware  constraints 

Asmptotic 

Schedule  realism 

Percent  added  cost  = percent  of  schedule 
acceleration 

Amount  of  previous  software  used 

Breakout  and  subjective 

Size,  structure  of  data  base 

Subjective 

Complexity 

Subjective 

Stability  of  requirements 

Subjective 

Stability  of  development  environment 

Subjective 

Representativeness  of  development  environ- 
ment 

Subjective 

Personnel 

Subjective;  approximately  5:1  variability 

Development  methods  (e.g.,  structured 
programming) 

Subjective;  systematic  approaches  cheaper 

Management 

Subjective:  high  variability 
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The  primary  step  in  using  this  model  is  the  determination  of  the 
number  of  delivered  executable  source  instructions  where  delivered  implies 
designed,  integrated,  tested,  and  document  (Reference  8).  Source  instruc- 
tions which  for  this  discussion  exclude  comment  cards,  is  considered  a 
better  estimation  factor  than  the  number  of  object  instructions  which  is 
then  used  in  the  Wolverton  and  Modified  Wolverton  models. 

Once  the  number  of  instructions  and  the  language  are  known,  cost 
factors  presented  in  Table  4 are  used  to  arrive  at  the  basic  cost  figure. 
As  can  be  seen  from  the  table,  many  factors  affect  the  cost  estimate, 
such  as  whether  it  is  a real-time  application  program,  familiar,  or 
unfamiliar  program,  etc. 

The  "relation  to  cost"  for  several  of  the  factors  identified  as 
invluencing  software  cost  are  listed  as  "subjective".  The  size  and 
structure  of  the  data  base  is  an  extremely  Important  parameter.  Quite 
naturally,  the  effect  on  cost  is  more  for  large  data  file  oriented 
projects  but  as  of  yet,  no  quantitative  relationship  similar  to  those 
developed  for  cost-per- instruction  has  been  established.  The  complexity 
factor  as  of  yet  has  not  been  defined  in  a way  so  as  to  be  used  reliably 
in  a cost  formula.  Attempts  have  been  made  to  correlate  costs  with  such 
factors  as  number  of  interfaces,  percentage  of  branch  statements,  and 
number  of  paths  through  a program,  but  without  any  highly  reliable 
correlations.  The  effect  on  cost  that  the  development  environment  has 
is  merely  the  added  cost  required  to  adapt  software  to  actual  operational 
conditions  such  as  different  computer  configuration  and  operating 
procedures;  can  be  quite  significant,  upwards  to  95  percent  in  some 
instances,  but  can  only  be  estimated  subjectively. 

Quality  of  personnel  is  considered  by  many  experienced  estimators 
to  be  the  most  important  factor  affecting  software  development  costs. 
Productivity  variations  of  5:1  between  individuals  are  common.  Yet  to  be 
developed  is  the  quantitative  effects  on  cost  of  using  development 
techniques  such  as  structured  programming,  top-down  development,  chief 
programmer  teams,  and  automated  aids.  It  is  agreed  that  systematic 
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approaches  to  software  development  are  better  than  disorganized  ones. 
Possible  payoffs  for  the  use  of  systematic  software  development  techniques 
are  in  opera+^ion  and  maintenance  costs  because  of  ease  of  debugging  and 
rebuilding  the  program. 

To  sum  up  the  ESD  approach,  the  basic  cost  is  arrived  at  by  utilizing 
the  number  of  instructions  times  the  cost  per  instruction  and  adding  cost 
for  type  of  program,  unfamiliar,  real-time,  etc.  Subjective  factors  are 
then  applied  to  adjust  cost  to  reflect  the  development  technique,  personel , 
etc . 

4.  THE  TECOLOTE  MODEL 

In  this  report,  the  Tecolote  Model  refers  to  the  basic  equations 
extracted  from  a report  entitled,  "A  Provisional  Model  for  Estimating 
Computer  Program  Development  Costs,"  Dec  1974,  (Reference  16)  prepared 
by  Brad  C.  Frederic  of  Tecolote  Research,  Inc.,  Santa  Barbara,  California, 
for  the  Resource  Analysis  Branch,  Office  of  the  Chief  of  Naval  Operations, 
Department  of  the  Navy  specifically  for  estimating  development  cost  for 
tactical  software.  Tactical  software  is  defined  by  Frederic  as  any 
complete  set  of  computer  programs  that  resides  in  and  drives  a computer 
system  within  a fire  control  system.  Mr.  Frederic  stressed  the  point 
that  the  model  was  a "provisional  model,"  that  is,  serving  only  for  the 
time  being. 

The  report  emphasized  the  problem  of  obtaining  data  to  perform 
statistical  analysis  and  noted  that  three  large  software  cost  data  bases 
had  been  already  compiled  at  System  Development  Corporation  (SDC),  TRW, 
and  North  American  Autonetics  (NAA).  There  were  problems  in  the  data 
collected  by  Tecolote  (387  separate  points  from  15  source  references) 
that  proved  insurmountable.  Since  the  data  had  to  be  collected  from 
rather  outdated  published  sources,  locating  spokesmen  familiar  with  the 
program  to  interpret  the  data  was  impossible.  Therefore,  the  data  base 
could  not  be  treated  or  rationalized  into  a homogeneous  base.  Hence, 
Tecolote  elected  to  undertake  a small  sample  approach  (5  data  points) 
utilizing  only  data  which  they  thoroughly  understood,  and  where  "the 
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estimating  relationships  developed  would  be  more  in  the  nature  of 
engineering  scaling  laws  than  strictly  derived  statistical  equations." 
(Reference  16) 

The  Tecolote  analysis  of  software  development  included  the  following 
activities,  as  given  by  Wolverton  (Section  V-1): 

a.  Software  requirements  generation. 

b.  Preliminary  software  design  (and  release). 

c.  Detailed  software  design  (and  release). 

d.  Code  and  debug. 

e.  Development  testing, 

f.  Validation  testing. 

g.  Operation  demonstration  (and  handover). 

The  types  of  computer  architectures  which  this  study  included  were 
single  Central  Processor  Units  (CPU),  democratic,  and  autocratic.  Single 
CPU  involves  a single  central  processor  with  storage  and  peripherals. 

The  democratic  architectures  consider  multiple  CPU's  operating  in  parallel 
with  pairwise  cormiunicatlon,  common  storage,  and  peripherals.  Autocratic 
is  a combination  of  a single  CPU's  and  democratic  subsystems  acting  in 
parallel,  under  the  control  of  a separate  single  CPU  executive. 

Mr,  Frederic  noted  that  computer  system  speed  and  fast  storage 
capacity  are  the  major  drivers  of  software  requirements.  The  size  of  the 
program  in  this  model  is  the  number  of  machine  language  instructions.  The 
size  can  be  input  as  either  the  number  of  operational  instructions  or  the 
number  of  delivered  instructions.  In  general,  the  number  of  delivered  is 
greater  than  the  number  of  operational  instructions.  Operational  instruc- 
tions are  those  produced  during  development  that  are  eventually  installed 
in  the  tactical  hardware;  delivered  instructions  are  all  those  instructions 
produced  during  development.  The  instructions  contained  in  a development 
"test  bed"  which  simulates  hardware  interfaces  are  an  example  of  delivered 
instructions  which  never  become  operational.  According  to  Frederic,  the 
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number  of  operational  instructions  increases  for  tactical  software  directly 
as  either  the  number  of  targets  terminal -tracked  increases  or  as  the  target 
approach  speed  increases. 

There  are  five  basic  cost  estimating  equations  derived  by  Tecolote 
given  in  Table  5.  Each  equation  requires  the  input  of  one  of  five  self 
explanatory  variables.  The  equation  which  the  user  utilizes  depends  on 
the  input  variable  which  he  is  more  confident  about.  For  example,  if  the 
user  knows  the  number  of  delivered  instructions  (D)  then  the  equation 
0.01(D)  ■ , 0 in  thousands,  results  in  total  development  cost.  Likewise, 

if  the  user  knows  number  of  operating  instructions  (0)  the  equation 
0.01(0)  ■ gives  you  total  development  costs.  Notes  A,  B,  and  C are 
helpful  in  terms  of  understanding  the  basic  assumptions  of  the  CER.  The 
output  is  the  total  development  cost  in  FY73  millions  of  dollars. 

5.  THE  IBM  MODEL 

The  IBM  Model  is  documented  in  the  IBM  proprietary  report  "Estimating 
Software  Life  Cycle  Costs:  by  John  C.  Malone,  April  1975  (Reference  17). 

The  report  utilized  software  cost  oata  which  was  derived  from  software 
projects  performed  by  IBM,  which,  (1)  employed  top-down  structured 
programming  techniques  and  (2)  utilized  the  Chief  Programmer  Teams 
Operation  Concept.  Structured  progranming  techniques  feature  a simple 
flow  of  logic  such  that  the  program  can  be  easily  read  and  understood. 
Structured  progranming  tends  to  improve  both  software  reliability  and 
maintainability  but  may  not  be  efficient  in  terms  of  computer  resource 
usage.  Structured  progranming  constrains  the  implementer  to  three  basic 
constructs,  "the  straight  line,"  "if  then  else,"  "do  while  (loop)." 

Top-down  progranming  is  starting  development  with  the  top  module  such 
that  the  real  driver  is  used  to  test  all  submodules  estimating  interface 
problems. 
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TABLE  5 

SUMMARY  OF  PROVISIONAL  SOFTWARE  ESTIMATING  RELATIONSHIPS  (SEE  NOTE  A) 
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NOTES:  (A)  COST  ASSUME  $3,930  FOR  LABOR,  $77/C0MPUTER  HOUR,  AND  4.23  COMPUTER  HOURS/MM. 

(B)  USE  AIR  THREAT  COLUMN  IF  MAXIMUM  THREAT  APPROACH  SPEED  IS  IN  THE  250-700  M/SEC  RANGE. 

(C)  USE  SEA  THREAT  COLUMN  IF  MAXIMUM  THREAT  APPROACH  SPEED  IS  50  M/SEC  OR  LESS. 
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The  chief  programmer  approach  depends  on  top-down  implementation, 
and  matches  personnel  capability  with  the  complexity  of  the  modules  they 
are  to  develop,  i.e.,  the  top-most  complex  modules  are  produced  by  a 
highly  qualified  software  system  specialist,  referred  to  as  the  chief 
programmer.  Less  qualified  personnel  implement  the  lower  lever  modules 
under  the  control  and  guidance  of  the  chief  programmer.  The  chief 
programmer  approach  to  software  implementation  is  a good  concept,  but  the 
staffing  profile  can  make  it  difficult  to  employ.  This  model  addresses 
only  the  software  development  phase.  The  data  included  costs  for  the 
development  phase  of  both  real-time  and  support  software.  The  equations 
being  of  a proprietary  nature  could  not  be  presented,  however,  the  results 
of  applying  the  model  are  presented  in  Section  VI  of  this  report. 

6.  NAVAL  AIR  DEVELOPMENT  CENTER  MODEL 

The  cost  relationship  (CER)  discussed  in  this  section  was  taken  from 
a study  done  by  Naval  Air  Development  Center  (NAVAIRDEVCEN  or  NADC) 
entitled,  "A  Cost  By  Function  Model  for  Avionic  Computer  Systems", 

March  1971  (Reference  13).  The  NAVAIRDEVCEN  developed  an  overall  CER, 
comprised  of  several  equations,  which  could  be  used  for  predicting  total 
acquisition  costs  for  research,  development,  test  and  evaluation,  and 
production  of  future  avionic  computer  systems.  Reference  13  gives  a 
complete  computer  listing  of  the  "Cost-by-Function"  model  with  its  10 
basic  modules.  These  10  basic  modules  are  as  follows: 

(1)  Raw  Technical  System  Requirements:  Functional  requirements  of 
the  system  are  translated  by  a function/structure  requirements  matrix  to 
six  variables  denoting  the  raw  technical  requirements  of  the  system. 

(2)  Total  Technical  System  Requirements:  The  raw  technical 
requirements  are  modified  using  system  architecture  factors  to  reflect 
performance  needed. 

(3)  Modularized  Technical  System  Requirements:  Converts  from  total 
technical  system  acquirements  to  integral  units  of  the  selected  hardware 
modules. 
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(4)  Cost  Trends  Near  Baseline:  Cost  trends  with  technical 
requirements  are  determined  in  the  vicinity  of  each  baseline.  This  module 
automatically  recalibrates  the  model  when  new  data  becomes  available. 

(5)  Programning  Costs  (RDT&E):  The  software  requirements  implied 
by  module  two  are  converted  to  RDT&E  programming  costs. 

(6)  Estimated  Hardware  Costs:  RDT&E  and  First  Unit  Production: 

By  utilizing  the  system  performance  characteristics,  baseline  characteristics 
and  cost  trends,  the  hardware  costs  are  estimated.  The  model  selects  a 
baseline  approximating  the  desired  system. 

'7)  Production  Cost  Breakout  by  Year:  A learning  curve  and  quantity 
disco  int  are  employed  and  aggregated  on  a yearly  basis  via  an  input 
production  schedule. 

(8)  Breakout  of  RDT&E  Hardware  Costs  by  Line  Item:  The  results  of 
module  six  are  broken  down  by  major  line  item. 

(9)  Breakdown  of  all  RDT&E  Costs  by  Year;  RDT&E  software  costs 

and  hardware  costs  are  broken  out  by  year  using  the  input  program  management 
factors. 

(10)  Summary  and  Report  Generation:  The  annual  programming  costs 
generated  by  module  five  and  the  production  cost  breakout  by  year,  module 
7,  are  summarized  and  a report  is  generated. 

The  following  equation,  referred  to  in  this  report  as  the  NAVAIRDEVCEN 
software  cost  model,  is  basically  module  five,  and  provides  an  estimate  of 
the  total  number  of  man-months  required  to  develop  a software  package  for 
an  avionic  computer  system: 

Y = 2.8X2  ^-^^3  * ^^^4  ■ ^^^5  ^7  ■ 
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where  Y = number  of  manmonths 

1 Xp  = number  of  machine  language  instructions  (thousands)  in  delivered 

^ program 

Xj  = number  of  man-miles  traveled  by  contractors 

X^  = number  of  document  types  produced 

Xj  = average  progratitner 's  experience  with  system  (NOTE:  The 
experience  "for  the  system  progranmer  is  the  sum  of  the  average  number  of 
years  of  experience  with  the  specific  computer -type,  application,  and 
language. " ) 

Xg  « number  of  independent  consoles 

X^  - percentage  of  new  instructions 

Module  two  can  be  used  to  calculate  the  variable  Xp  based  on  the 
function  of  the  program.  An  interesting  cotiment  found  in  a GRC  report 
notes  that  the  weighing  factor  applied  to  the  documentation  in  this  model 
‘ is  based  upon  pre-work  standard  (WS)  8506  experience.  GRC,  based  upon  a 

private  communication,  31  Aug  73,  states  that  the  documentation  costs  can 
be  expected  to  triple  with  the  implementation  of  WS  8506.  Hence,  the  X^ 
term  would  be  modified  and  the  equation  would  appear  as: 

Y * 2.8X,  + 1.3X,  + 99X.  - 17Xc  + lOX,  + X,  - 188  (2) 

2 3 4 5 6 7 ' ' 

7.  AEROSPACE  MODEL 

The  model  referred  to  here  as  the  "Aerospace  Model"  was  taken  from 
a 1975  Aerospace  Corporation  report  on  cost  estimating  (Reference  18). 

The  data  used  to  develop  the  cost  equations  for  this  model  were  divided 
into  two  groups  or  types  of  programming  efforts,  real-time  programs,  and 
support  programs.  Included  in  the  cost  data  are  costs  that  accrued  as  a 
result  of  problems  encountered  in  developing  a large-scale  software 
program.  The  real-time  software  program  development  problem  areas 
identified  were: 

a.  Limited  core  storage  of  computers. 

b.  Timing  requirements 
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c.  Accuracy  requirements, 

d.  Fixed-point  arithmetic. 

e.  Changing  specifications. 

f.  Real-time  simulations. 

1.  Inability  to  interface  languages. 

2.  Nonstandardization  of  computers  between  machines  and 
operational  program  or  support  program  problem  areas  identified  were: 

(a)  Timing  and  accuracy  problems. 

(b)  Inability  to  transfer  simulation  activities  of  one 
contractor  to  another  due  to  language  and  machine  differences. 

(c)  Inadequate  and  changing  specifications. 

(d)  Lack  of  an  organized  method  of  defining  endpoints  and 
products  of  various  development  phases. 

The  data  base  used  to  develop  the  cost  equation  for  real-time  software 
program  costs  consisted  of  13  large-scale  programs,  primarily  airborne  and 
space  oriented  programs.  The  cost  equation  derived  from  a regression 
analysis  of  those  13  data  points.  The  cost  equation  developed  is  as  follows: 

Man-months  = 0.057  (Instruction)^'^^  (3) 

The  sample  size  for  operational  support  programs  consisted  of  seven 
data  points  (both  airborne  and  ground  software  programs  were  in  the  data 
base).  The  resulting  equation  for  support  software  man-months  estimation 
is: 

Man-Months  = 2.012  (Instruction)®’^®^  (4) 

The  coiment  about  language  type  mentioned  above  holds  true  in  this  case 
as  wel 1 , 
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Once  the  number  of  man-months  required  for  development  is  estimated 
using  Equation  3 or  4,  a dollar  value  per  man-month  is  used  to  derive  the 
total  development  cost.  The  estimated  cost  per  man-month  would  obviously 
vary  with  the  particular  company  performing  the  programming  function.  For 
planning  purposes,  an  average  of  $5,000  per  man-month  is  used  by  Aerospace 
Corp. 

8.  GENERAL  RESEARCH  CORPORATION  MODEL 

The  GRC  model  was  taken  from  a report  entitled  "Estimation  of 
Computer  Requirements  and  Software  Development  Costs",  March  1974, 
prepared  by  M.  A.  Taback  and  M.  C.  Ditmore  of  General  Research  Corporation 
(Reference  12).  The  purpose  of  GRC  was  to  determine  a means  of  quantifying 
computer  software  development  cost  from  overall  system  requirements.  GRC 
had  previously  developed  a procedure  for  determining  the  data  processing 
speed  and  memory  required  to  implement  various  computer  functions  from 
system  performance  requirements.  The  report  presents  a cost  estimating 
relationship  for  computer  software  development  which  models  the  effects 
of  the  following;  (1)  program  size,  (2)  computer  language,  (3)  complexity, 
and  (4)  hardware  constraints.  The  key  conclusion  of  the  report  was  that 
the  program  size,  used  along  with  the  effects  of  program  complexity, 
high-level  language,  and  hardware  constraints,  is  a reasonable  predictor 
of  software  development  cost. 

The  first  step  in  utilizing  any  of  the  GRC  models  or  any  other  such 
model,  that  of  estimating  the  number  of  instructions  for  the  particular 
program,  appears  to  be  the  critical  step.  GRC  suggests  that  one  should 
develop  the  algorithms  that  are  required  and  then  utilize  Table  6,  which 
is  a table  of  typical  functional  requirements  in  terms  of  number  of 
instructions  required,  to  implement  the  algorithm. 

The  CER  developed  by  GRC  used  the  factors  which  they  felt  could  be 
identified  either  prior  to  program  start  up  or  iimediately  thereafter. 

The  major  factors  include: 

a.  Estimated  number  of  instructions. 

b.  Language  used. 
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TABLE  6 

FUNCTIONAL  TRENDS  IN  AVIONICS  MEMORY  REQUIREMENTS 


Typical  Current  Applications 

Instructions  and 

Constants* 

Navigation 

2600 

Air-to-Air  Weapon  Delivery 

930 

Air-to-Ground  Weapon  Delivery 

1120 

Data  Link 

630 

Tacan  & Steering 

120 

Radar  Update 

45. 

Attitude  Data 

380 

Displays  and  Control 

1100 

Self  Test 

570 

Executive  4 Input/Output 

1400 

Cotimon  Subroutines 

300 

NADC  Guidel ines** 

Instructions  and 

Data 

Radar  Processing 

16,000 

Acoustical  Processing 

16,000 

ASW  Non-Acoustical  Sensors 

16,000 

Navigation 

8,000 

FI ight  Control 

8,000 

Data  Collection 

8,000 

Fire  Control 

8,000 

Recon  Data 

8,000 

Display  Processing 

16,000 

Data  Conmuni cations 

8,000 

Console  and  Cockpit 

16,000 

Projection  for  Near-Future  Bomber 

Instructions  and 

Da  ta 

Navigation 

17,000 

Weapon  Delivery 

9,000 

Target/Check  Point  Acquisition 

4,500 

Radar  Homing,  Location 

14,500 

Conmunications 

10,000 

Countermeasures 

5,000 

Mission  Data  Center 

11,500 

Controls,  Display  & Outputs 

17,000 

Miscellaneous 

7,000 

^Constants  are  fixed  numbers  that  are  used  In  computations  and  are 
prestored,  along  with  the  instructions  that  use  them. 

**Minimum  requirements,  main  storage  only  (offline  mass  storage 
estimated  separately). 
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c.  Degrer  of  difficulty  or  complexity. 

d.  Degree  of  saturation  of  the  host  computer,  where  degree  of 
saturation  refers  to  the  amount  of  excess  central  processor  speed  and 
memory  storage  available  to  the  programmer.  The  resulting  CER  provides 
the 


= 0.232  N.  (5) 

total  software  development  cost  in  1973  dollars  assuming  unlimited 
computer  resources,  where  = total  software  development  cost  and  N.  = 
number  of  machine  language  instructions  in  the  software  development 
effort  under  consideration.  Equation  5 does  not  include  the  extra  cost 
incurred  in  the  development  of  an  operating  system  for  a new  computer 
or  the  modification  of  an  existing  operating  system  to  accommodate  the 
software  program.  Development  cost  for  compilers,  assemblers,  and  other 
support  software  must  be  handled  as  additional  software  to  be  developed. 


Addressing  software  developments  within  the  context  of  constrained 
computer  resources,  if  we  let  P = fraction  of  maximum  speed  and  memory 
capacity  utilized  the  total  constrained  software  cost,  (C^lc  is 


(CJc  = pMr.-.  for  P > 0.5 

" ^ 1 


(6) 


If,  for  example,  a system  were  to  utilize  75  percent  of  its  memory 
capacity  (P  = 0.75)  then  the  CER  reduces  to: 


(Cs)c  = 


0.7 


1 -JolT 


= 1.40  C, 


(7) 


GRC  cited  three  primary  effects  of  the  use  of  a higher  order 
language  (HOL)  and  stressed  the  fact  that  these  are  a first  approximation 
of  the  effect  of  an  HOL: 


1.  One  to  three  times  as  much  storage  space  is  required  for  a 
HOL  as  for  a machine  oriented  language  (MOL),  depending  on  the  type  of 
language  and  the  compiler  used. 
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2.  Execution  of  a HOL  is  one  to  three  times  slower  than  execution 
of  MOL,  depending  on  type  of  compiler. 

3.  Programming  costs  for  a HOL  are  one-half  to  one-third  those  of 

MOL. 

9.  THE  SYSTEM  DEVELOPMENT  CORPORATION  MODEL 

The  System  Development  Corporation  (SDC)  in  1967  published  a report 
entitled,  "Management  Handbook  for  the  Estimation  of  Computer  Progratming 
Costs,"  based  on  work  sponsored  by  the  ESD  (Reference  19).  This  report 
includes  qualitative  discussions/guidelines  to  help  managers  estimate 
costs  of  computer  programming.  SDC  spent  considerable  time  analyzing  a 
large  amount  of  data  in  an  attempt  to  identify  the  dominant  factors 
impacting  programming  costs.  Table  7 presents  the  distribution  of 
software  programs  in  the  SDC  data  base  by  programming  application. 

Through  regression  analysis  on  the  94  variables  displayed  in  Table  8, 
SDC  identified  12  variables  which  were  sufficiently  significant  to  use 
as  estimating  indices.  For  this  analysis  105  programs  categorized  as  the 
large  computer  subsample  were  selected  from  their  software  data  base. 

The  large  computer  subsample  consisted  of  software  developed  for  machines 
with  a monthly  rental  price  or  equivalent  purchase  price  of  $750,000  or 
greater.  Equation  8 estimates  man-months  per  thousand  instructions  coded 
(Y)  expressed  in  terms  of  the  12  variables  identified. 

Y = 0.049  + 15.2Xg  -0.23X25  + O.528X3Q  + 4.50X37  + 0.091X^g 
- 17.5X^g  + 25.1X5^  + 22.0X54  + 26.0X55  - 0.25Xg4  - 14.9Xg5 

+ 10.4X74  (8) 


where: 

Xg  = Complexity  of  the  program  system  interface.  In  the  computer 
program,  if  more  than  50  percent  of  the  design  effort  is  devoted  to 
problems  associated  with  transferring  data  to  or  from  the  program  data 
point,  Xg  = 2;  if  between  10  percent  and  50  percent  effort  is  devoted  to 
data  transfer  problems,  Xg  = 1 ; if  less  than  10  percent  effort  is  devoted, 
Xg-O. 
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TABLE  8 


COMPUTER  SOFTWARE  VARIABLES  (REFERENCE  19) 

Vagueness  of  design  requirements  definition. 

Innovation  required. 

Lack  of  knowledge  of  operational  requirements. 

Number  of  organizational  users. 

Number  of  ADP  centers. 

Complexity  of  program  system  interface. 

Response  time  requirements. 

Stabil ity  of  design. 

On-line  requirements. 

Total  object  instructions  delivered. 

Percent  delivered  object  instructions  reused. 

Total  nondelivered  object  instructions  produced. 

Total  source  instructions  written. 

Percent  source  instructions  written  in  POL  (Procedure  Oriented 
Language) . 

Percent  of  total  source  instructions  discarded. 

Percent  of  total  object  instructions  discarded. 

Number  of  conditional  branches. 

Number  of  words  in  the  data  base. 

Number  of  classes  of  items  in  the  data  base. 

Number  of  input  message  types. 

Number  of  output  message  types. 

Number  of  input  variables. 

Number  of  output  variables. 
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TABLE  8 (Cont'd) 

Number  of  words  in  tables,  and  constants  not  in  data  base. 
Percent  clerical  instructions. 

Percent  mathematical  instructions. 

Percent  input/output  instructions. 

Percent  logical  control  instructions. 

Percent  self-checking  instructions. 

Percent  information  storage  and  retrieval  functions. 
Percent  data  acquisition  and  display  function. 

Percent  control  or  regulation  function. 

Percent  decision-making  functions. 

Percent  transformation  functions. 

Percent  generation  functions. 

Average  operating  time. 

Frequency  of  operation. 

Insufficient  memory. 

Insufficient  I/O  capacity. 

Stringent  timing  requirements. 

Number  of  subprograms. 

Programming  language. 

POL  expansion  ratio. 

Support  program  availability. 

Internal  documentation. 

External  documentation. 

Total  number  of  document  types. 

Type  of  program  (business,  scientific,  utility,  other) 
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TABLE  8 (Conf  d) 


Compiler  or  assembler  used. 

Developmental  computer  used. 

First  program  on  computer. 

Average  turn-around  time. 

ADP  components  developed  concurrently. 

Special  display  equipment. 

Core  capacity. 

Random  access  device  used. 

Number  of  bits  per  word. 

Memory  access  time. 

Machine  add  time. 

Compute  cost. 

Percent  senior  progratmers. 

Average  programmer  experience  with  language. 

Average  programmer  experience  with  application. 

Percent  programmers  participating  in  program  design. 

Personnel  continuity. 

Maximum  number  of  programmers. 

Lack  of  management  procedures. 

Number  of  agencies  concurring  in  design. 

Customer  inexperience. 

Computer  operated  by  agency  other  than  program  developer. 

Program  developed  at  site  other  than  the  operational  installation. 
Different  computers  for  prograirniing  and  operation. 
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TABLE  8 (Concluded) 

Closed  or  open  shop  operation. 

Number  of  locations  for  program  data  point  development. 

X^^  Number  of  man  trips. 

X^g  Program  data  point  developed  by  military  organization. 

Program  data  point  developed  on  time-shared  computer. 

X^g  Complexity  of  system  interface  with  other  systems. 

X^g  Security  classification  level. 

XgQ  Number  of  sources  of  system  information. 

Xg.|  Accessibility  of  system  information. 

^82  Degree  of  system  change  expected  during  development. 

Xgg  Degree  of  system  change  expected  during  system  operations. 

Xg^  Number  of  functions  in  the  system. 

Xgg  Number  of  system  components. 

Xgg  Number  of  components  --  not  off-the-shelf. 

Xg^  Percent  senior  analysts. 

Xgg  Quality  of  resource  documents. 

Xgg  The  availability  of  special  tools. 

Xqq  Degree  of  standardization  in  policy  and  procedures. 

Xg^  Number  of  official  reviews  of  documents. 

Xg2  Personnel  turnover. 

Xgg  Output  volume. 

Xg^  Input  volume. 
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^25  " clerical  instructions. 

X30  = Percent  of  information  storage  and  retrieval  functions. 

= Frequency  of  operation.  If  this  variable  is  not  applicable, 
X37  = 0;  if  frequency  of  operation  is  less  than  one  per  month,  = 1 
more  than  one  per  month  and  less  than  one  per  week,  = 2;  more  than 

one  per  week  and  less  than  one  per  day,  X3^  = 3;  if  daily,  X3^  = 4;  if 
utility  or  on-line,  (including  compilers)  X3^  = 5. 

X^g  = External  documentation.  This  is  the  number  of  pages  written 
for,  or- distributed  to,  customers. 


X^g  ^ = Business.  For  programs  classified  as  business  application, 
X^8  1=1;  the  remaining  applications,  X^g  ^ = 0. 

Xgi  = First  program  on  computer.  If  it  is  a new  machine  or  new  to 
the  inst.^'llation  and  to  the  prograrrmers,  Xg^  = 1.  If  old  or  not  new. 


Xg^  = Special  display  equipment  involving  use  of  graphic  displays, 
CRTs,  scopes,  etc.  Xg^  = 1 if  used,  Xg^  = 0 if  not  used. 


Xgg  = Random  access  device  used  such  as  drum,  disc,  etc.  Xgg  = 1 


if  used. 


Xgg  = 0 if  not  used. 


XgA  = Percent  programmers  participating  in  program  design.  This 
is  the  ratio  of  programmers  participating  in  the  design  of  the  program 
to  the  total  number  of  programmers  assigned  to  the  program  development. 


Xgg  “ Personnel  continuity,  specifically,  the  number  of  personnel 
working  for  the  duration  of  the  project  divided  by  the  maximum  number 
assigned  at  any  time. 


X^^  = Number  of  locations  for  program  development. 


As  GRC  points  out  in  Reference  12,  this  equation  requires  rather 
detailed  previous  knowledge  of  software  parameters,  and  when  such 
information  is  unavailable  or  cannot  be  estimated  with  accuracy,  the 
technique  cannot  be  used. 
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SECTION  VI 

APPLICATION  OF  MODELS 

All  but  one  of  the  models  presented  In  Section  V of  this  report 
were  used  to  analyze  cost  associated  with  six  large  scale  computer 
programs.  Because  of  the  lack  of  sufficient  data,  the  SDC  model  was 
not  applicable.  Actual  expended  costs  were  known  on  two  of  the  six 
software  packages  analyzed.  Although  this  exercise  should  not  be 
considered  as  a validation  or  even  an  evaluation  of  the  models,  the 
results  may  assist  software  managers  in  becoming  sensitive  and  aware  of 
what  factors  affect  such  engineering  economic  estimates. 

The  six  programs  will  be  described  in  as  much  detail  as  was  needed 
to  apply  the  CER’s  and  the  basic  assumptions  that  were  made  will  be 
presented.  Input  data  requirements  for  the  eight  models  under 
consideration  are  presented  in  Table  9.  Table  10  displays  input  data 
assumed  for  this  exercise  for  each  of  the  software  development  efforts 
analyzed. 

1.  DIGITAL  AVIONICS  INFORMATION  SYSTEM  (DAIS) 

The  Digital  Avionics  Information  System  (DAIS)  Air  Superiority 
Program  contained  36,916  statements  and  was  prograirmed  in  a higher 
order  language,  JOVIAL  (J73/I).  The  program  was  broken  down  into  nine 
('  modules  (executive,  navigation,  weapon  delivery,  ECM,  control/display, 

flight  control,  management,  communications,  DAIS  Integrated  Test  System) 
by  number  of  instructions  and  type  of  program,  i.e.,  real-time, 
operating  system,  utility,  or  application.  The  following  facts  and 
assumptions  were  made  pertaining  to  each  module:  (1)  executive  is  an 
operating  system  containing  4000  instructions,  (2)  navigation  containing 
3848  instructions  was  real-time  program  application  type,  (3)  weapon 
delivery  is  a real-time  program  application  type  containing  3272 
instructions,  (4)  ECM  contained  534  Instructions  and  is  a real-time 
application  type  program,  (5)  control/display  containing  3663  instructions 
is  a real-time  utility  program,  (6)  flight  control,  real-time  application 
program  includes  8770  instructions,  (7)  management  containing  7838 
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TABLE  9 

SOFTWARE  COST  ESTIMATE  INPUT  WORKSHEET,  DEFINITION  OF 
TERMS,  AND  SOFTWARE  OUTPUT  SHEET 

LANGUAGE  TYPE (HOL  or  MOL) 

If  HOL,  give  name  

ESTIMATE  NUMBER  OF  TOTAL  INSTRUCTIONS  

BREAKOUT  OF  TYPE  OF  INSTRUCTIONS: 

# of  control  instructions  

# pre-post  CPU  instructions  

# algorithm  instructions 

# data  management  instructions 

# real  time  instructions  

DELIVERED  OR  OPERATING  INSTRUCTIONS?  

OBJECT  OR  SOURCE  INSTRUCTIONS?  

NEW  OR  OLD  PROGRAM  (%  new,  % old)?  

' RANK  DIFFICULTY  (between  1 and  9,  1 = easy,  9 = hard)  

MANRATED  OR  NON-MANRATED?  

IF  UNFAMILIAR  PROGRAM  (NEW),  RANK  BETWEEN  1.5  and  2.0 
(Subjective  ranking  of  how  unfamiliar  program  is) 

OPERATING  SYSTEM  OR  NOT?  

REAL  TIME  SYSTEM  OR  NOT  (%)?  

# MAN  MILES  TRAVELED  BY  CONTRACTOR  

# DOC  TYPES  

SYSTEM  PROGRAMMER  EXPERIENCE  (YEARS)  

# INDEPENDENT  CONSOLES  

# SUBROUTINES  

# REUSEABLE  SUBROUTINES  

FUNCTION  OF  PROGRAM  (BRIEF  STATEMENT)  

f 

1 


I 
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TABLE  9 (Cont'd) 


LANGUAGE  - Self  explanatory 

NUMBER  OF  INSTRUCTIONS  - Self  explanatory 

CONTROL  INSTRUCTION  - Controls  execution  flow  and  is  non-time  critical. 

PRE-POST  CPU  INSTRUCTION  - Pre-  or  post  algorithm  processor  which  manipulates 
data  for  subsequent  processing  or  output. 

ALGORITHM  INSTRUCTION  - Which  performs  logical  or  mathematical  operations 

DATA  MANAGEMENT  INSTRUCTIONS  - Data  management  routine  which  manages  data 
transfer  within  the  computer. 

REAL-TIME  INSTRUCTIONS  - Time  critical  processor  which  is  highly  optimized 
machine  dependent  code. 

DELIVERED  OR  OPERATING  INSTRUCTIONS  - Delivered  is  the  total  of  instructions 
received  from  contractor  as  opposed  to  the  actual  instructions  you  would  use 
(operating).  Contractor  may  have  to  simulate  your  machine  (system). 

OBJECT  VS  SOURCE  INSTRUCTIONS  - Object  is  machine  language  instructions  after 
source  deck  has  been  compiled.  Source  ■>  compiler  -*■  object 

NEW  OR  OLD  - Self  explanatory 

RANK  DIFFICULTY  - Subjective  l=easy  5=median  9=difficult 
MANRATED  - NON-MANRATED  - Self  explanatory 

UNFAMILIAR  - For  unfamiliar,  multiply  by  1.5  - 2.0.  A judgement  of  how 
unfamiliar  the  program  is  to  the  progranner. 

OPERATING  SYSTEM  - Software  which  controls  the  execution  of  computer 
programs  and  which  may  provide  scheduling,  debugging,  I/O  control,  accounting, 
compilation,  storage  assignment,  data  management,  and  related  services. 
Operating  system  program  component,  of  a system,  costs  more  per  instruction, 
than  the  application  or  utility  program  components. 

REAL  TIME  - Real  time  programs  are  those  in  which  the  time  is  kept  as  a 
variable,  stored  in  memory,  to  be  incremented  or  stepped  under  program 
control.  It  Is  used  to  describe  processes  in  which  the  computer  is 
controlling  a device  and  must  receive  input  signals  and  transmit  output 
signals  within  the  certain  maximum  time.  For  examplj,  SAT.  control,  ship 
control,  flight  control,  navigation. 

# MAN-MILES  TRAVELED  - Miles  per  man  traveled  by  the  contractor  to  and 
from  the  customer. 

# DOC  TYPES  - Reports,  flow  charts,  user  manuals,  etc. 

SYSTEM  PROGRAMMER  EXP.  - Total  years  of  experience  with  the  particular 
system. 

# SUBROUTINES  - Self  explantory 

# OF  REUSED  SUBROUTINES  - Reused  from  previous  programs. 
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TABLE  9 (Concluded) 


SOFTWARE  OUTPUT  SHEET 


(Name  of  Software  Program) 


TOTAL  COST  TO  DEVELOP  PROGRAM  ($75) 


COST  OF  ANALYSIS  '$ 

DESIGN  $ 

CODE  $ 

TEST  $ 

DOCUMENTATION  $' 


TOTAL  MAN  MTHS  (P  $3930/MAN  MTH) 
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instruction  is  an  operating  system,  (8)  communications  containing  9991 
instructions  is  a real-time  application,  and  (9)  DITS  is  a utility  program 
of  4000  instructions.  The  total  DAIS  air  superiority  program  was  man-rated. 

2.  DAIS  CLOSE  AIR  SUPPORT 

The  DAIS  Close  Air  Support  program  contained  37,169  statements.  It 
was  also  programned  in  JOVIAL  (J73/I)  and  broken  down  into  nine  modules 
(Executive,  Navigation,  Weapon  Delivery,  ECM,  Control/Display,  Flight  Control, 
Management,  Communications,  Subroutines)  by  number  of  instructions  and  type 
of  program,  i.e.,  real-time,  operating  system,  utility,  or  application. 

The  following  assumptions  were  made  pertaining  to  each  module:  (1)  executive 
containing  8000  instructions  was  an  operating  system,  (2)  the  navigation, 
weapon  delivery,  ECM,  flight  control,  and  communication  modules  are  all 
real-time  application  programs  with  5939,  6534,  534,  496,  and  991  instructions 
respectively,  (3)  the  Control/Display  module  is  a utility  real-time  program 
of  3663  instruction,  (4)  management  containing  7838  instruction  is  an 
operating  system,  and  (5)  Subroutines  contain  3174  instructions  and  was  a 
utility  program. 

For  both  the  DAIS  Air  Superiority  and  Close  Air  Support  programs,  a 
subjective  "point  on  the  learning  curve"  of  1.7  was  assumed.  This  number 
is  based  on  the  ESD  model  (Table  4)  where  "if  unfamiliar,  multiply  by  1.5 
to  2.0."  (Reference  8) 

/ r 

3. ^  F-15  JOINT  TACTICAL  INFORMATION  DISTRIBUTION  SYSTEM  (JTIDS) 

The  F-15  Joint  Tactical  Information  Distribution  System  (JTIDS) 
software  program  has  two  versions,  one  for  minicomputers,  one  for  micro- 
computers. The  information  about  these  programs  was  gathered  from 
Reference  15.  Both  the  mini-  and  micro-computer  JTIDS  software  programs 
consisted  of  12  modules.  Table  11  displays  the  basic  information  about 
each  module.  More  detailed  information  is  contained  in  the  reference. 
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TABLE  11 


JTIDS  PROGRAM  BREAKDOWN 


Name 

Wol verton 
Mini  Cost 

Instructions  Category 

Instructions 

Operating 
System  or 

Real -T ime 

Executive 

1000 

T 

1450 

Operating 

Mate  Subroutine 

876 

OM/A 

1270 

Neither 

Self  Test 

1271 

NM/P 

1719 

Neither 

Input  Message  Buffer 

25 

NM/P 

36 

Neither 

Message  Type  Filter 

180 

NM/P 

261 

Nei ther 

Data  Base  Management 

7020 

NM/P 

7155 

Operating  Sys 

Prompter 

120 

NM/P 

174 

Real-time 

Situation  Display 

1340 

NM/P 

1745 

Real-time 

Received  Command  Processing 

3620 

NM/P 

4349 

Real-time 

Acknowledge  Function 

60 

NM/P 

83 

Real-time 

Message  Construction 

466 

NM/P 

579 

Neither 

Parameter  Storage 

450 

NM/P 

450 

Neither 

Both  mini-  and  micro-computer  JTIDS  programs  utilized  the  higher  ordered 
languages  of  FORTRAN  or  JOVIAL. 

4.  DAIS  SUPPORT  SOFTWARE 

The  last  two  programs  to  be  analyzed  were  the  DAIS  J73/I  compiler  and 
the  Software  Design  and  Verification  System  (SDVS).  Both  the  J73/I  compiler 
and  the  SDVS  programs  are  good  data  points  since  we  have  actual  costs  to 
compare  the  estimates  against. 
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The  J73/I  compiler  consisted  of  132,087  source  instructions,  written 
in  HOL.  It  was  an  old  program  which  was  modified,  and  ranked  8.5  in 
difficulty  (1  = easy,  10  = hard).  The  program  was  a man-rated,  nonoperating 
system,  and  non-real-time  program.  Total  man-miles  traveled  by  contractor 
was  68,400.  The  number  of  document  types  was  five,  with  three  on-line, 
remote  terminals,  and  50  percent  of  the  program  being  new  instructions. 

The  average  of  the  programmers'  experience  was  assumed  to  be  three  years. 

The  Software  Design  and  Verification  System  (SDVS)  was  also  written 
in  J73/1  and  had  82,965  source  instructions.  SDVS  was  a non-man-rated, 
operating  system  program  with  9.0  degree  difficulty.  The  program  was 
non-real -time  with  eight  document  types.  An  average  of  21,600  miles  was 
traveled  by  contractors  with  an  average  of  2.8  years  software  experience. 

The  number  of  on-line  remote  terminals  is  five  with  100  percent  new 
instructions. 

5.  RESULTS 

Table  12  is  a summary  chart  of  results  obtained  by  applying  the  eight 
software  cost  estimating  models.  For  application  of  those  models,  which 
were  not  formulated  in  terms  of  the  RCA  PRICE  model,  economic  inflation 
rates  were  utilized  to  adjust  the  results  to  1977  dollars.  Those  cases 
where  the  information  obtained  was  insufficient  to  utilize  a particular 
methodology  are  identified  on  Table  12. 
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SECTION  VII 

SUPPORT/MAINTENANCE  COST  AREA 

Once  the  software  program  has  been  accepted,  continual  support  must 
be  furnished  to  modify  the  software  package  to  meet  changing  mission  and 
performance  requirements.  Besides  the  modifications  to  software  programs, 
corrections  must  be  made  to  previously  undetected  errors  which  occur. 

Because  software  is  the  controlling  and  integrating  agent  in  weapon  systems 
today,  proper  support  is  required  to  insure  that  the  program  performs  its 
intended  functions  properly.  In  avionic  software  programs  this  support  is 
critical  sinc“  errors  could  result  in  inadvertent  armament  release  or 
impact  with  the  ground  in  terrain  following  modes. 

The  software  development  process  is  typically  oriented  toward 
minimizing  the  total  development  time  ''r  maximizing  the  program's  efficiency. 

In  a study  on  the  relative  amount  of  time  spent  on  software  maintenance  it 
was  shown  that  most  software  facilities  spent  somewhere  between  20  and  30 
percent  of  their  time  on  software  maintenance,  but  some  installations 
spent  90  to  100  percent  of  their  time  maintaining  software.  Air  Force 
avionics  software  is  much  like  the  latter  and  "currently  it  costs  something 
like  $75  per  instruction  to  develop  the  software,  but  the  maintenance  of 
the  software  has  cos^  up  to  $4,000  per  instruction."  (Reference  20). 

Further  noted  by  Judith  A.  Clapp  (The  MITRE  Corp)  in  "A  Review  of  Software 
Cost  Estimation  Methods"  (Reference  18)  was  the  fact  that  54  percent  of 
all  errors  were  found  after  acceptance  tests  were  conducted  and  of  these 
84  percent  were  design  errors;  also,  of  the  total  number  er-^ors  found, 

64  percent  were  attributed  to  mistakes  in  design.  Throughout  the  development 
phase  relatively  little  thought  is  usually  given  about  what  will  happen 
after  development  is  completed.  According  to  the  CCIP-85  Report  (Reference  5), 
three  things  are  likely  to  happen  after  development: 

(1)  Another  organization  will  want  to  use  all  or  part  of  the  software 
for  its  application,  (2)  the  user  will  upgrade  eventually  to  a new  machine 
and  will  wish  to  convert  the  software,  and  (3)  users  will  quite  frequently 
want  the  programs  changed  to  meet  new  requirements,  produce  new  reports, 
accommodate  new  inputs,  clear  up  inconsistencies,  add  new  options,  etc. 
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Software  transferability  involves  addressing  the  ease  with  which  the 
first  two  points  mentioned  above  can  be  accomplished.  Maintainability, 
quite  simply  stated,  involves  the  capabilities  to  satisfy  the  last  point. 
Both  the  transferability  and  maintainability  aspects  involve  considerable 
costs  and  inconveniences.  A couple  of  prime  examples  of  the  costs  involved 
with  transferability  and  maintainability  were  given  in  the  CCIP-85  Report: 
Strategic  Air  Command  (SAC)  estimated  it  would  take  three  years  for  200 
programmers  to  convert  the  SAC  Control  System  (SACCS)  software  to  the 
upcoming  SAC  Worldwide  Military  Command  and  Control  System  computer.  This 
is  equivalent  to  three  years  worth  of  delays  and  roughly  $30  million  in 
costs.  It  took  150  programmers  one  year  to  convert  software  for  Electronic 
Intelligence  (ELINT)  and  Minuteman  application  onto  the  IBM  360/85; 
currently  the  360/85  nas  about  75  maintenance  programmers.  The  PACER 
software  cost  $8  million  to  develop  and  is  maintained  by  about  50  pro- 
granmers,  which  is  an  annual  software  maintenance  cost  of  about  25  percent 
of  development  costs.  Conversion  and  maintenance  expenses  could  be 
reduced  by  such  things  as:  machine-independent,  problem-oriented  pro- 
gramming languages;  use  of  structured  programming  techniques;  development 
of  computer  software  maintenance  and  transfer  aids;  maintenance  of  an 
Air  Force  software  library;  and  formulation  of  a standard  for  computer 
hardware,  software,  terminology  and  documentation.  "Reduction  in 
proliferation  of  different  computer  hardware  and  software  styles  would 
reduce  the  high  cost  of  retraining,  particularly  considering  Air  Force 
officer  rotation  policies:  The  Keesler  Training  Center  spent  $9.6  million 
in  training  computer  analysts,  prograirmers,  operators,  and  maintenance 
personnel  in  FY69."  (Reference  5) 

A search  of  current  literature  results  in  very  little  in  the  way  of 
predicting  support  and  maintenance  cost  for  computer  software.  Unlike 
hardware  operation  and  support  (O&S)  models,  where  the  cost  of  spares, 
maintenance  manhours,  materials,  training,  etc.,  can  be  estimated  based 
on  some  physical  characteristics  of  the  system,  software  maintenance  is 
strictly  a function  of  manhours  to  perform  the  necessary  actions.  Thus 
far,  maintenance  costs  for  software  seem  to  be  primarily  an  engineering 
estimate  by  an  expert,  someone  familiar  with  the  changes  to  be  made  to 
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a program,  rather  than  putting  certain  parameters  into  an  CER  or  formula 
and  calculating  annual  maintenance  casts.  The  effects  of  the  structured 
programming  or  chief  programner  approaches  on  maintenance  costs  can  only 
be  subjectively  estimated  as  of  this  time. 

The  "Aerospace  Model",  was  discussed  in  Section  V-7,  is  a total  life 
cycle  cost  model.  The  procedure  permits  costs  for  design  and  development 
(D&D),  investment,  and  operations  and  maintenance  (O&M)  to  be  determined 
in  a series  of  prearranged  steps.  "The  model  first  calculates  hardware 
(CPU)  costs,  then  applies  factors  for  estimating  the  other  D&D,  investment, 
and  O&M  costs,  and  finally  summarizes  the  total  program  costs."  (Reference 
21).  Most  of  the  factors  in  the  model  were  developed  based  on  a report 
entitled,  "Investment  Costs  for  Flight  Area  Defense  Systems,"  also  referred 
to  as  the  FADS  study.  The  primary  maintenance  equations  for  software 
appear  as  follows: 

Software  training  costs  during  production  phase: 

Initial  Civilian  = number  of  men  X 27,200 

Initial  Contractor  = number  of  men  X 35,598 

Initial  Military  = number  of  men  X 17,400 

b.  During  the  deployment  phase: 

Personnel  contractor  support  cost  = (number  of  men)  X $48,000  X 
(number  of  years  O&M  or  deplo^mient) 

Military  support  cost  = (number  of  men)  X $18,000  X (number  of 
years  O&M  or  deployment) 

These  equations  should  only  be  used  if  the  estimator  has  no  prior  basis 
for  determining  costs  of  any  of  his  data-processing  system  elements.  The 
model,  as  mentioned  above,  calculates  hardware  and  software  costs,  and  is 
referred  to  as  a "Data  Processing  System  Cost  Model". 
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In  the  pdst,  software  support  has  been  performed  by  the  contractor 
long  after  transition  to  the  user.  Because  of  the  nature  of  most  airborne 
software  systems,  the  contractor  alone  had  the  expertise  and  equipment  to 
perform  these  follow  on  maintenance  actions  and  modification.  "Large 
operational  flight  programs  (OFP's)  usually  are  never  absolutely  debugged, 
and  errors  can  remain  undetected  for  long  periods  of  time  due  to  the 
extremely  large  number  of  logic  paths."  (Reference  5).  Interesting  to 
note  at  this  time  is  that  currently  the  largest  program  which  has  been 
mathematically  proven  correct  has  about  400  instructions;  on  the  other 
hand,  experience  with  the  SAC  Control  System  (SACCS)  program,  with  about 
2.7  million  instructions,  indicates  that  about  one  software  error  per  day 
is  discovered  (Reference  5).  Figure  3 suimiar’zes  current  experience  in 
software  meantime  between  failure  (MTBF)  in  days  as  a function  of  program 
size  in  instructions. 


Figure  3.  Program  Size  vs  Reliability 
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The  Air  Force  has  recognized  the  dependence  upon  contractors  for 
maintenance  support  and  has  taken  steps  to  develop  in-house  capabilities 
to  support  current  and  future  OFP's.  Two  recent  studies  of  software 
management,  F-111  and  A-10,  show  promising  results  of  Air  Force  in-house 
capabil ities. 

To  combat  the  maintenance  costs  in  1974  work  began  on  an  Avionics 
Integration  and  Support  Facility  (AISF)  at  the  Sacramento  Air  Logistics 
Center  (ALC).  All  ALC's  software  programs  are  to  be  supported  by  AISF. 

The  AISF  facility  will  provide  hardware/ software  integration  support  as 
well  as  a dynamic  simulation  facility.  The  purpose  of  the  dynamic 
simulator  is  to  execute  the  OFP's  in  a hit  by  bit  fashion  to  debug  the 
software  functions. 

The  large  computer  at  AISF  will  make  it  easier,  safer,  and  less 
expensive  than  flight  tests  to  validate  OFP's.  Not  only  will  the  AISF 
facility  support  OFP  data  processing,  technical  data,  and  procedure 
verification,  but  also  provide  air  crew  familiarization  and  training. 

The  AISF  is  costing  approximately  $20  million  for  development  and 
implementation.  There  are  presently  40  contractors  amd  60  Air  Force 
military  and  civilian  engineers  working  at  the  AISF  facility.  Estimated 
annual  recurring  costs  for  the  facility  are  about  $3  million  (Reference  15). 

Since  there  is  a lack  of  fully  validated  O&M  predictive  models  for 
software  programs,  (Aerospace,  and  the  forthcoming  GRC  "LCC  software 
model"  are  the  only  "existing"  models  at  the  present  time),  this  AISF 
facility  can  hopefully  provide  a point  of  contact  to  supply  O&M  cost 
estimates  for  software  programs  under  development. 
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APPENDIX  A 

This  appendix  contains  the  computer  program  listing  of  the  Modified 
Wolverton  model.  The  next  two  pages  contain  the  listing  with  the  remaining 
pages  containing  an  example  output  listing.  The  following  inputs  are 
required: 

NC  = Number  of  control  routine  instructions 
NIO  * Number  of  input/output  instructions 

NP  = Number  of  Pre-  or  Post-  algorithm  processor  instructions 
NA  = Number  of  algorithm  instructions 
ND  = Number  of  data  management  instructions 
NT  = Number  of  time  critical  processor  instructions 
K 

The  above  six  inputs  were  defined  in  Section  V-1,  and  utilized  the 
following  format  statement:  12  F0RMAT(615) 

For  the  example  the  following  values  were  used: 

NC=0  NI0=0  NP='1964  NA=5702  ND=7155  NT=1450 
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